Skip to the content.

ismir2004songs

This is the tempo_eval report for the ‘ismir2004songs’ corpus.

Reports for other corpora may be found here.

Table of Contents

References for ‘ismir2004songs’

References

1.0

Attribute Value
Corpus ismir2004songs
Version 1.0
Curator Fabien Gouyon
Data Source manual annotation
Annotation Tools derived from beat annotations
Annotation Rules median of inter beat intervals
Annotator, bibtex Gouyon2006
Annotator, ref_url http://mtg.upf.edu//ismir2004/contest/tempoContest/node6.html

Basic Statistics

Reference Size Min Max Avg Stdev Sweet Oct. Start Sweet Oct. Coverage
1.0 465 23.00 242.44 89.83 27.84 56.00 0.74

Table 1: Basic statistics.

CSV JSON LATEX PICKLE

Smoothed Tempo Distribution

Figure 1: Percentage of values in tempo interval.

CSV JSON LATEX PICKLE SVG PDF PNG

Estimates for ‘ismir2004songs’

Estimators

alonso2004/ismir2004_corr

Attribute Value
Corpus ismir2004songs
Version 1.0
Data Source ISMIR 2004 Songs contest results, http://www.iua.upf.edu/mtg/ismir2004/contest/tempoContest/data3.tar.gz
Annotation Tools AlonsoACF, see also: Fabien Gouyon, Anssi P. Klapuri, Simon Dixon, Miguel Alonso, George Tzanetakis, Christian Uhle, and Pedro Cano. An experimental comparison of audio tempo induction algorithms. IEEE Transactions on Audio, Speech, and Language Processing, 14(5):1832\u2013 1844, 2006.
Annotator, bibtex Alonso2004

alonso2004/ismir2004_spec

Attribute Value
Corpus ismir2004songs
Version 1.0
Data Source ISMIR 2004 Songs contest results, http://www.iua.upf.edu/mtg/ismir2004/contest/tempoContest/data3.tar.gz
Annotation Tools AlonsoSP, see also: Fabien Gouyon, Anssi P. Klapuri, Simon Dixon, Miguel Alonso, George Tzanetakis, Christian Uhle, and Pedro Cano. An experimental comparison of audio tempo induction algorithms. IEEE Transactions on Audio, Speech, and Language Processing, 14(5):1832\u2013 1844, 2006.
Annotator, bibtex Alonso2004

boeck2015/tempodetector2016_default

Attribute Value
Corpus ismir2004songs
Version 0.17.dev0
Annotation Tools TempoDetector.2016, madmom, https://github.com/CPJKU/madmom
Annotator, bibtex Boeck2015

davies2009/mirex_qm_tempotracker

Attribute Value  
Corpus ismir2004songs  
Version 1.0  
Annotation Tools QM Tempotracker, Sonic Annotator plugin. https://code.soundsoftware.ac.uk/projects/mirex2013/repository/show/audio_tempo_estimation/qm-tempotracker Note that the current macOS build of ‘qm-vamp-plugins’ was used.  
Annotator, bibtex Davies2009 Davies2007

dixon2001/ismir2004_indu

Attribute Value
Corpus ismir2004songs
Version 1.0
Data Source ISMIR 2004 Songs contest results, http://www.iua.upf.edu/mtg/ismir2004/contest/tempoContest/data3.tar.gz
Annotation Tools DixonI, see also: Fabien Gouyon, Anssi P. Klapuri, Simon Dixon, Miguel Alonso, George Tzanetakis, Christian Uhle, and Pedro Cano. An experimental comparison of audio tempo induction algorithms. IEEE Transactions on Audio, Speech, and Language Processing, 14(5):1832\u2013 1844, 2006.
Annotator, bibtex Dixon2001

dixon2001/ismir2004_trac

Attribute Value
Corpus ismir2004songs
Version 1.0
Data Source ISMIR 2004 Songs contest results, http://www.iua.upf.edu/mtg/ismir2004/contest/tempoContest/data3.tar.gz
Annotation Tools DixonT, see also: Fabien Gouyon, Anssi P. Klapuri, Simon Dixon, Miguel Alonso, George Tzanetakis, Christian Uhle, and Pedro Cano. An experimental comparison of audio tempo induction algorithms. IEEE Transactions on Audio, Speech, and Language Processing, 14(5):1832\u2013 1844, 2006.
Annotator, bibtex Dixon2001

dixon2003/ismir2004_auco

Attribute Value
Corpus ismir2004songs
Version 1.0
Data Source ISMIR 2004 Songs contest results, http://www.iua.upf.edu/mtg/ismir2004/contest/tempoContest/data3.tar.gz
Annotation Tools DixonACF, see also: Fabien Gouyon, Anssi P. Klapuri, Simon Dixon, Miguel Alonso, George Tzanetakis, Christian Uhle, and Pedro Cano. An experimental comparison of audio tempo induction algorithms. IEEE Transactions on Audio, Speech, and Language Processing, 14(5):1832\u2013 1844, 2006.
Annotator, bibtex Dixon2003

echonest/version_3_2_1

Attribute Value
Corpus ismir2004songs
Version 3.2.1
Data Source Graham Percival and George Tzanetakis. Streamlined tempo estimation based on autocorrelation and crosscorrelation with pulses. IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP), 22(12):1765–1776, 2014.
Annotation Tools Echo Nest track analyzer v3.2.1
Annotator, bibtex Percival2014

gkiokas2012/default

Attribute Value
Corpus ismir2004songs
Version 1.0
Data Source Graham Percival and George Tzanetakis. Streamlined tempo estimation based on autocorrelation and crosscorrelation with pulses. IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP), 22(12):1765–1776, 2014.
Annotation Tools Gkiokas2012
Annotator, bibtex Gkiokas2012

klapuri2006/ismir2004

Attribute Value
Corpus ismir2004songs
Version 1.0
Data Source ISMIR 2004 Songs contest results, http://www.iua.upf.edu/mtg/ismir2004/contest/tempoContest/data3.tar.gz
Annotation Tools Klapuri, see also: Fabien Gouyon, Anssi P. Klapuri, Simon Dixon, Miguel Alonso, George Tzanetakis, Christian Uhle, and Pedro Cano. An experimental comparison of audio tempo induction algorithms. IEEE Transactions on Audio, Speech, and Language Processing, 14(5):1832\u2013 1844, 2006.
Annotator, bibtex Klapuri2006

klapuri2006/percival2014

Attribute Value
Corpus ismir2004songs
Version 1.0
Data Source Graham Percival and George Tzanetakis. Streamlined tempo estimation based on autocorrelation and crosscorrelation with pulses. IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP), 22(12):1765–1776, 2014.
Annotation Tools Klapuri 2006
Annotator, bibtex Klapuri2006

oliveira2010/ibt

Attribute Value
Corpus ismir2004songs
Version 1.0
Data Source Graham Percival and George Tzanetakis. Streamlined tempo estimation based on autocorrelation and crosscorrelation with pulses. IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP), 22(12):1765–1776, 2014.
Annotation Tools Oliveira 2010
Annotator, bibtex Oliveira2010

percival2014/stem

Attribute Value
Corpus ismir2004songs
Version 1.0
Annotation Tools percival 2014, ‘tempo’ implementation from Marsyas, http://marsyas.info, git checkout tempo-stem
Annotator, bibtex Percival2014

scheirer1998/ismir2004

Attribute Value
Corpus ismir2004songs
Version 1.0
Data Source ISMIR 2004 Songs contest results, http://www.iua.upf.edu/mtg/ismir2004/contest/tempoContest/data3.tar.gz
Annotation Tools Scheirer, see also: Fabien Gouyon, Anssi P. Klapuri, Simon Dixon, Miguel Alonso, George Tzanetakis, Christian Uhle, and Pedro Cano. An experimental comparison of audio tempo induction algorithms. IEEE Transactions on Audio, Speech, and Language Processing, 14(5):1832– 1844, 2006.
Annotator, bibtex Scheirer1998

scheirer1998/percival2014

Attribute Value
Corpus ismir2004songs
Version 1.0
Data Source Graham Percival and George Tzanetakis. Streamlined tempo estimation based on autocorrelation and crosscorrelation with pulses. IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP), 22(12):1765–1776, 2014.
Annotation Tools Scheirer 1998
Annotator, bibtex Scheirer1998

schreiber2014/default

Attribute Value
Corpus ismir2004songs
Version 0.0.1
Annotation Tools schreiber 2014, http://www.tagtraum.com/tempo_estimation.html
Annotator, bibtex Schreiber2014

schreiber2017/ismir2017

Attribute Value
Corpus ismir2004songs
Version 0.0.4
Annotation Tools schreiber 2017, model=ismir2017, http://www.tagtraum.com/tempo_estimation.html
Annotator, bibtex Schreiber2017

schreiber2017/mirex2017

Attribute Value
Corpus ismir2004songs
Version 0.0.4
Annotation Tools schreiber 2017, model=mirex2017, http://www.tagtraum.com/tempo_estimation.html
Annotator, bibtex Schreiber2017

schreiber2018/cnn

Attribute Value
Corpus ismir2004songs
Version 0.0.2
Data Source Hendrik Schreiber, Meinard Müller. A Single-Step Approach to Musical Tempo Estimation Using a Convolutional Neural Network. In Proceedings of the 19th International Society for Music Information Retrieval Conference (ISMIR), Paris, France, Sept. 2018.
Annotation Tools schreiber tempo-cnn (model=cnn), https://github.com/hendriks73/tempo-cnn

schreiber2018/fcn

Attribute Value
Corpus ismir2004songs
Version 0.0.2
Data Source Hendrik Schreiber, Meinard Müller. A Single-Step Approach to Musical Tempo Estimation Using a Convolutional Neural Network. In Proceedings of the 19th International Society for Music Information Retrieval Conference (ISMIR), Paris, France, Sept. 2018.
Annotation Tools schreiber tempo-cnn (model=fcn), https://github.com/hendriks73/tempo-cnn

schreiber2018/ismir2018

Attribute Value
Corpus ismir2004songs
Version 0.0.2
Data Source Hendrik Schreiber, Meinard Müller. A Single-Step Approach to Musical Tempo Estimation Using a Convolutional Neural Network. In Proceedings of the 19th International Society for Music Information Retrieval Conference (ISMIR), Paris, France, Sept. 2018.
Annotation Tools schreiber tempo-cnn (model=ismir2018), https://github.com/hendriks73/tempo-cnn

sun2021/default

Attribute Value
Corpus ismir2004songs
Version 0.0.2
Data Source Xiaoheng Sun, Qiqi He, Yongwei Gao, Wei Li. Musical Tempo Estimation Using a Multi-scale Network. in Proc. of the 22nd Int. Society for Music Information Retrieval Conf., Online, 2021
Annotation Tools https://github.com/Qqi-HE/TempoEstimation_MGANet
Annotator, bibtex Sun2021

tzanetakis2002/ismir2004_hist

Attribute Value
Corpus ismir2004songs
Version 1.0
Data Source ISMIR 2004 Songs contest results, http://www.iua.upf.edu/mtg/ismir2004/contest/tempoContest/data3.tar.gz
Annotation Tools TzanetakisH, based on front-end described in Tzanetakis2002, see also: Fabien Gouyon, Anssi P. Klapuri, Simon Dixon, Miguel Alonso, George Tzanetakis, Christian Uhle, and Pedro Cano. An experimental comparison of audio tempo induction algorithms. IEEE Transactions on Audio, Speech, and Language Processing, 14(5):1832– 1844, 2006.
Annotator, bibtex Tzanetakis2002

tzanetakis2002/ismir2004_mmul

Attribute Value
Corpus ismir2004songs
Version 1.0
Data Source ISMIR 2004 Songs contest results, http://www.iua.upf.edu/mtg/ismir2004/contest/tempoContest/data3.tar.gz
Annotation Tools TzanetakisMM, based on front-end described in Tzanetakis2002, see also: Fabien Gouyon, Anssi P. Klapuri, Simon Dixon, Miguel Alonso, George Tzanetakis, Christian Uhle, and Pedro Cano. An experimental comparison of audio tempo induction algorithms. IEEE Transactions on Audio, Speech, and Language Processing, 14(5):1832– 1844, 2006.
Annotator, bibtex Tzanetakis2002

tzanetakis2002/ismir2004_msum

Attribute Value
Corpus ismir2004songs
Version 1.0
Data Source ISMIR 2004 Songs contest results, http://www.iua.upf.edu/mtg/ismir2004/contest/tempoContest/data3.tar.gz
Annotation Tools TzanetakisMs, based on front-end described in Tzanetakis2002, see also: Fabien Gouyon, Anssi P. Klapuri, Simon Dixon, Miguel Alonso, George Tzanetakis, Christian Uhle, and Pedro Cano. An experimental comparison of audio tempo induction algorithms. IEEE Transactions on Audio, Speech, and Language Processing, 14(5):1832– 1844, 2006.
Annotator, bibtex Tzanetakis2002

uhle2004/ismir2004

Attribute Value
Corpus ismir2004songs
Version 1.0
Data Source ISMIR 2004 Songs contest results, http://www.iua.upf.edu/mtg/ismir2004/contest/tempoContest/data3.tar.gz
Annotation Tools Uhle, see also: Fabien Gouyon, Anssi P. Klapuri, Simon Dixon, Miguel Alonso, George Tzanetakis, Christian Uhle, and Pedro Cano. An experimental comparison of audio tempo induction algorithms. IEEE Transactions on Audio, Speech, and Language Processing, 14(5):1832– 1844, 2006.
Annotator, bibtex Rohden2004

zplane/auftakt_v3

Attribute Value
Corpus ismir2004songs
Version 3.0
Data Source Graham Percival and George Tzanetakis. Streamlined tempo estimation based on autocorrelation and crosscorrelation with pulses. IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP), 22(12):1765–1776, 2014.
Annotation Tools zplane aufTAKT version 3.0, http://licensing.zplane.de/technology#auftakt
Annotator, bibtex Percival2014

Basic Statistics

Estimator Size Min Max Avg Stdev Sweet Oct. Start Sweet Oct. Coverage
alonso2004/ismir2004_corr 465 57.78 245.89 131.76 49.76 92.00 0.60
alonso2004/ismir2004_spec 459 50.06 249.17 106.80 45.47 61.00 0.59
boeck2015/tempodetector2016_default 465 45.11 240.00 120.93 37.09 79.00 0.76
davies2009/mirex_qm_tempotracker 465 65.42 246.09 126.75 31.62 84.00 0.86
dixon2001/ismir2004_indu 465 50.09 248.80 127.38 53.03 91.00 0.51
dixon2001/ismir2004_trac 448 54.97 248.47 157.87 44.05 121.00 0.79
dixon2003/ismir2004_auco 465 60.00 249.00 163.38 44.85 124.00 0.81
echonest/version_3_2_1 465 53.59 198.29 99.03 30.86 64.00 0.76
gkiokas2012/default 464 45.00 248.00 114.30 36.35 76.00 0.74
klapuri2006/ismir2004 465 66.00 164.00 111.45 21.14 79.00 0.96
klapuri2006/percival2014 464 68.45 164.06 110.83 20.63 76.00 0.97
oliveira2010/ibt 465 81.00 161.00 115.72 21.86 81.00 1.00
percival2014/stem 465 51.55 159.01 101.36 22.21 70.00 0.89
scheirer1998/ismir2004 465 51.50 220.00 106.08 35.85 66.00 0.69
scheirer1998/percival2014 416 61.35 181.82 104.23 27.96 77.00 0.79
schreiber2014/default 465 43.49 161.46 97.19 21.57 68.00 0.89
schreiber2017/ismir2017 465 48.81 200.14 98.49 23.27 68.00 0.87
schreiber2017/mirex2017 465 39.09 185.87 90.89 26.18 61.00 0.75
schreiber2018/cnn 465 43.00 224.00 109.80 31.02 76.00 0.80
schreiber2018/fcn 465 45.00 224.00 106.12 33.87 68.00 0.73
schreiber2018/ismir2018 465 54.00 224.00 112.95 28.96 77.00 0.86
sun2021/default 465 47.00 238.00 110.69 32.89 84.00 0.77
tzanetakis2002/ismir2004_hist 465 50.00 249.00 172.55 75.11 125.00 0.66
tzanetakis2002/ismir2004_mmul 465 52.00 240.00 140.89 45.67 113.00 0.67
tzanetakis2002/ismir2004_msum 465 50.00 248.00 135.23 53.03 117.00 0.55
uhle2004/ismir2004 465 51.00 248.00 119.77 41.23 77.00 0.74
zplane/auftakt_v3 464 65.90 164.50 109.22 23.58 79.00 0.89

Table 2: Basic statistics.

CSV JSON LATEX PICKLE

Smoothed Tempo Distribution

Figure 2: Percentage of values in tempo interval.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy

Accuracy1 is defined as the percentage of correct estimates, allowing a 4% tolerance for individual BPM values.

Accuracy2 additionally permits estimates to be wrong by a factor of 2, 3, 1/2 or 1/3 (so-called octave errors).

See [Gouyon2006].

Note: When comparing accuracy values for different algorithms, keep in mind that an algorithm may have been trained on the test set or that the test set may have even been created using one of the tested algorithms.

Accuracy Results for 1.0

Estimator Accuracy1 Accuracy2
schreiber2017/mirex2017 0.8989 0.9484
schreiber2014/default 0.7312 0.9183
schreiber2018/fcn 0.6753 0.9355
schreiber2017/ismir2017 0.6688 0.9140
schreiber2018/cnn 0.6452 0.9226
percival2014/stem 0.6086 0.8602
sun2021/default 0.5871 0.9054
schreiber2018/ismir2018 0.5871 0.9183
klapuri2006/ismir2004 0.5828 0.9097
klapuri2006/percival2014 0.5785 0.8925
echonest/version_3_2_1 0.5699 0.7849
gkiokas2012/default 0.5677 0.9075
zplane/auftakt_v3 0.5613 0.8237
boeck2015/tempodetector2016_default 0.5548 0.9484
oliveira2010/ibt 0.4667 0.7677
davies2009/mirex_qm_tempotracker 0.4387 0.8194
scheirer1998/percival2014 0.4258 0.6430
uhle2004/ismir2004 0.4194 0.7204
scheirer1998/ismir2004 0.3785 0.6946
alonso2004/ismir2004_spec 0.3742 0.6860
dixon2001/ismir2004_indu 0.2860 0.6280
tzanetakis2002/ismir2004_msum 0.2731 0.5247
alonso2004/ismir2004_corr 0.2344 0.5785
tzanetakis2002/ismir2004_hist 0.2129 0.4774
dixon2001/ismir2004_trac 0.1935 0.6882
tzanetakis2002/ismir2004_mmul 0.1892 0.4129
dixon2003/ismir2004_auco 0.1677 0.7677

Table 3: Mean accuracy of estimates compared to version 1.0 with 4% tolerance ordered by Accuracy1.

CSV JSON LATEX PICKLE

Raw data Accuracy1: CSV JSON LATEX PICKLE

Raw data Accuracy2: CSV JSON LATEX PICKLE

Accuracy1 for 1.0

Figure 3: Mean Accuracy1 for estimates compared to version 1.0 depending on tolerance.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy2 for 1.0

Figure 4: Mean Accuracy2 for estimates compared to version 1.0 depending on tolerance.

CSV JSON LATEX PICKLE SVG PDF PNG

Differing Items

For which items did a given estimator not estimate a correct value with respect to a given ground truth? Are there items which are either very difficult, not suitable for the task, or incorrectly annotated and therefore never estimated correctly, regardless which estimator is used?

Differing Items Accuracy1

Items with different tempo annotations (Accuracy1, 4% tolerance) in different versions:

1.0 compared with alonso2004/ismir2004_corr (356 differences): ‘Abba/Gold - Greatest Hits/20 sec/01-Dancing Queen’ ‘Abba/Gold - Greatest Hits/20 sec/02-Knowing Me, Knowing You’ ‘Abba/Gold - Greatest Hits/20 sec/03-Take A Chance On Me’ ‘Abba/Gold - Greatest Hits/20 sec/06-Super Trouper’ ‘Abba/Gold - Greatest Hits/20 sec/08-The Winner Takes It All’ ‘Abba/Gold - Greatest Hits/20 sec/10-S. O. S’ ‘Abba/Gold - Greatest Hits/20 sec/11-Chiquitita’ ‘Abba/Gold - Greatest Hits/20 sec/12-Fernando’ ‘Abba/Gold - Greatest Hits/20 sec/14-Gimme! Gimme! Gimme! (A Man After Midnight)’ ‘Abba/Gold - Greatest Hits/20 sec/15-Does Your Mother Know’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/01-A_Dream_Within_A_Dream’ … CSV

1.0 compared with alonso2004/ismir2004_spec (291 differences): ‘Abba/Gold - Greatest Hits/20 sec/01-Dancing Queen’ ‘Abba/Gold - Greatest Hits/20 sec/03-Take A Chance On Me’ ‘Abba/Gold - Greatest Hits/20 sec/04-Mamma Mia’ ‘Abba/Gold - Greatest Hits/20 sec/08-The Winner Takes It All’ ‘Abba/Gold - Greatest Hits/20 sec/09-Money, Money, Money’ ‘Abba/Gold - Greatest Hits/20 sec/10-S. O. S’ ‘Abba/Gold - Greatest Hits/20 sec/12-Fernando’ ‘Abba/Gold - Greatest Hits/20 sec/17-The Name Of The Game’ ‘Abba/Gold - Greatest Hits/20 sec/19-Waterloo’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/02-The_Raven’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/04-The_Cask_Of_Amontillado’ … CSV

1.0 compared with boeck2015/tempodetector2016_default (207 differences): ‘Abba/Gold - Greatest Hits/20 sec/11-Chiquitita’ ‘Abba/Gold - Greatest Hits/20 sec/16-One Of Us’ ‘Abba/Gold - Greatest Hits/20 sec/17-The Name Of The Game’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/03-The_Tell-tale_Heart’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/04-The_Cask_Of_Amontillado’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/05-_The_System_Of__Doctor_Tarr_And_Professor_Fether’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/09-The_Fall_Of_The_House_Of_Usher__IV__Pavane’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/01-Tarrega_ Recuerdos de la Alhambra’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/02-Lauro_ Dos Valses Venezolanos - Uno’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ … CSV

1.0 compared with davies2009/mirex_qm_tempotracker (261 differences): ‘Abba/Gold - Greatest Hits/20 sec/08-The Winner Takes It All’ ‘Abba/Gold - Greatest Hits/20 sec/11-Chiquitita’ ‘Abba/Gold - Greatest Hits/20 sec/12-Fernando’ ‘Abba/Gold - Greatest Hits/20 sec/16-One Of Us’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/02-The_Raven’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/03-The_Tell-tale_Heart’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/04-The_Cask_Of_Amontillado’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/05-_The_System_Of__Doctor_Tarr_And_Professor_Fether’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/07-The_Fall_Of_The_House_Of_Usher__II__Arrival’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/11-To_One_In_Paradise’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/01-Tarrega_ Recuerdos de la Alhambra’ … CSV

1.0 compared with dixon2001/ismir2004_indu (332 differences): ‘Abba/Gold - Greatest Hits/20 sec/01-Dancing Queen’ ‘Abba/Gold - Greatest Hits/20 sec/08-The Winner Takes It All’ ‘Abba/Gold - Greatest Hits/20 sec/09-Money, Money, Money’ ‘Abba/Gold - Greatest Hits/20 sec/11-Chiquitita’ ‘Abba/Gold - Greatest Hits/20 sec/16-One Of Us’ ‘Abba/Gold - Greatest Hits/20 sec/17-The Name Of The Game’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/02-The_Raven’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/03-The_Tell-tale_Heart’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/04-The_Cask_Of_Amontillado’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/05-_The_System_Of__Doctor_Tarr_And_Professor_Fether’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/11-To_One_In_Paradise’ … CSV

1.0 compared with dixon2001/ismir2004_trac (375 differences): ‘Abba/Gold - Greatest Hits/20 sec/01-Dancing Queen’ ‘Abba/Gold - Greatest Hits/20 sec/02-Knowing Me, Knowing You’ ‘Abba/Gold - Greatest Hits/20 sec/03-Take A Chance On Me’ ‘Abba/Gold - Greatest Hits/20 sec/06-Super Trouper’ ‘Abba/Gold - Greatest Hits/20 sec/09-Money, Money, Money’ ‘Abba/Gold - Greatest Hits/20 sec/10-S. O. S’ ‘Abba/Gold - Greatest Hits/20 sec/11-Chiquitita’ ‘Abba/Gold - Greatest Hits/20 sec/12-Fernando’ ‘Abba/Gold - Greatest Hits/20 sec/14-Gimme! Gimme! Gimme! (A Man After Midnight)’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/01-A_Dream_Within_A_Dream’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/02-The_Raven’ … CSV

1.0 compared with dixon2003/ismir2004_auco (387 differences): ‘Abba/Gold - Greatest Hits/20 sec/01-Dancing Queen’ ‘Abba/Gold - Greatest Hits/20 sec/02-Knowing Me, Knowing You’ ‘Abba/Gold - Greatest Hits/20 sec/03-Take A Chance On Me’ ‘Abba/Gold - Greatest Hits/20 sec/06-Super Trouper’ ‘Abba/Gold - Greatest Hits/20 sec/07-I Have A Dream’ ‘Abba/Gold - Greatest Hits/20 sec/08-The Winner Takes It All’ ‘Abba/Gold - Greatest Hits/20 sec/11-Chiquitita’ ‘Abba/Gold - Greatest Hits/20 sec/14-Gimme! Gimme! Gimme! (A Man After Midnight)’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/03-The_Tell-tale_Heart’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/04-The_Cask_Of_Amontillado’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/07-The_Fall_Of_The_House_Of_Usher__II__Arrival’ … CSV

1.0 compared with echonest/version_3_2_1 (200 differences): ‘Abba/Gold - Greatest Hits/20 sec/08-The Winner Takes It All’ ‘Abba/Gold - Greatest Hits/20 sec/16-One Of Us’ ‘Abba/Gold - Greatest Hits/20 sec/17-The Name Of The Game’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/04-The_Cask_Of_Amontillado’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/01-Tarrega_ Recuerdos de la Alhambra’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/04-Buzzing’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/06-Modern_Apprentice’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/07-Operation_Eagle_Lie’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/08-Hypocrite’ … CSV

1.0 compared with gkiokas2012/default (201 differences): ‘Abba/Gold - Greatest Hits/20 sec/16-One Of Us’ ‘Abba/Gold - Greatest Hits/20 sec/17-The Name Of The Game’ ‘Abba/Gold - Greatest Hits/20 sec/19-Waterloo’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/03-The_Tell-tale_Heart’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/04-The_Cask_Of_Amontillado’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/05-_The_System_Of__Doctor_Tarr_And_Professor_Fether’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/09-The_Fall_Of_The_House_Of_Usher__IV__Pavane’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/01-Tarrega_ Recuerdos de la Alhambra’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/02-Lauro_ Dos Valses Venezolanos - Uno’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ … CSV

1.0 compared with klapuri2006/ismir2004 (194 differences): ‘Abba/Gold - Greatest Hits/20 sec/16-One Of Us’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/03-The_Tell-tale_Heart’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/04-The_Cask_Of_Amontillado’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/07-The_Fall_Of_The_House_Of_Usher__II__Arrival’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/09-The_Fall_Of_The_House_Of_Usher__IV__Pavane’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/11-To_One_In_Paradise’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/01-Tarrega_ Recuerdos de la Alhambra’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/02-Lauro_ Dos Valses Venezolanos - Uno’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/01-Assassin’ … CSV

1.0 compared with klapuri2006/percival2014 (196 differences): ‘Abba/Gold - Greatest Hits/20 sec/16-One Of Us’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/03-The_Tell-tale_Heart’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/04-The_Cask_Of_Amontillado’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/07-The_Fall_Of_The_House_Of_Usher__II__Arrival’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/09-The_Fall_Of_The_House_Of_Usher__IV__Pavane’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/11-To_One_In_Paradise’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/01-Tarrega_ Recuerdos de la Alhambra’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/02-Lauro_ Dos Valses Venezolanos - Uno’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/01-Assassin’ … CSV

1.0 compared with oliveira2010/ibt (248 differences): ‘Abba/Gold - Greatest Hits/20 sec/08-The Winner Takes It All’ ‘Abba/Gold - Greatest Hits/20 sec/11-Chiquitita’ ‘Abba/Gold - Greatest Hits/20 sec/12-Fernando’ ‘Abba/Gold - Greatest Hits/20 sec/16-One Of Us’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/03-The_Tell-tale_Heart’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/04-The_Cask_Of_Amontillado’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/07-The_Fall_Of_The_House_Of_Usher__II__Arrival’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/11-To_One_In_Paradise’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/01-Tarrega_ Recuerdos de la Alhambra’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/02-Lauro_ Dos Valses Venezolanos - Uno’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ … CSV

1.0 compared with percival2014/stem (182 differences): ‘Abba/Gold - Greatest Hits/20 sec/11-Chiquitita’ ‘Abba/Gold - Greatest Hits/20 sec/16-One Of Us’ ‘Abba/Gold - Greatest Hits/20 sec/17-The Name Of The Game’ ‘Abba/Gold - Greatest Hits/20 sec/19-Waterloo’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/03-The_Tell-tale_Heart’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/04-The_Cask_Of_Amontillado’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/01-Tarrega_ Recuerdos de la Alhambra’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/02-Lauro_ Dos Valses Venezolanos - Uno’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/01-Assassin’ … CSV

1.0 compared with scheirer1998/ismir2004 (289 differences): ‘Abba/Gold - Greatest Hits/20 sec/03-Take A Chance On Me’ ‘Abba/Gold - Greatest Hits/20 sec/04-Mamma Mia’ ‘Abba/Gold - Greatest Hits/20 sec/06-Super Trouper’ ‘Abba/Gold - Greatest Hits/20 sec/08-The Winner Takes It All’ ‘Abba/Gold - Greatest Hits/20 sec/09-Money, Money, Money’ ‘Abba/Gold - Greatest Hits/20 sec/13-Voulez Vous’ ‘Abba/Gold - Greatest Hits/20 sec/15-Does Your Mother Know’ ‘Abba/Gold - Greatest Hits/20 sec/16-One Of Us’ ‘Abba/Gold - Greatest Hits/20 sec/19-Waterloo’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/02-The_Raven’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/03-The_Tell-tale_Heart’ … CSV

1.0 compared with scheirer1998/percival2014 (267 differences): ‘Abba/Gold - Greatest Hits/20 sec/11-Chiquitita’ ‘Abba/Gold - Greatest Hits/20 sec/13-Voulez Vous’ ‘Abba/Gold - Greatest Hits/20 sec/16-One Of Us’ ‘Abba/Gold - Greatest Hits/20 sec/19-Waterloo’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/02-The_Raven’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/04-The_Cask_Of_Amontillado’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/05-_The_System_Of__Doctor_Tarr_And_Professor_Fether’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/11-To_One_In_Paradise’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/02-Lauro_ Dos Valses Venezolanos - Uno’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ … CSV

1.0 compared with schreiber2014/default (125 differences): ‘Abba/Gold - Greatest Hits/20 sec/16-One Of Us’ ‘Abba/Gold - Greatest Hits/20 sec/17-The Name Of The Game’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/03-The_Tell-tale_Heart’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/04-The_Cask_Of_Amontillado’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/09-The_Fall_Of_The_House_Of_Usher__IV__Pavane’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/01-Tarrega_ Recuerdos de la Alhambra’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/02-Lauro_ Dos Valses Venezolanos - Uno’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/04-Buzzing’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/13-Real_Areas_For_Investigation’ … CSV

1.0 compared with schreiber2017/ismir2017 (154 differences): ‘Abba/Gold - Greatest Hits/20 sec/08-The Winner Takes It All’ ‘Abba/Gold - Greatest Hits/20 sec/16-One Of Us’ ‘Abba/Gold - Greatest Hits/20 sec/17-The Name Of The Game’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/03-The_Tell-tale_Heart’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/04-The_Cask_Of_Amontillado’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/09-The_Fall_Of_The_House_Of_Usher__IV__Pavane’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/02-Lauro_ Dos Valses Venezolanos - Uno’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/04-Buzzing’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/07-Operation_Eagle_Lie’ … CSV

1.0 compared with schreiber2017/mirex2017 (47 differences): ‘Abba/Gold - Greatest Hits/20 sec/08-The Winner Takes It All’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/03-The_Tell-tale_Heart’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/13-Real_Areas_For_Investigation’ ‘Bach - Walcha/DGG 419 047-2/20sec/04-Praludium u. Fuge f-moll BWV 534 - II_ Fuge’ ‘Bach/20 sec/Bach - Cello suite 2 d - 01 Prelude’ ‘Bebel_Gilberto/Tanto_Tempo/20sec/04-Sem_Contencao’ ‘Bebel_Gilberto/Tanto_Tempo/20sec/07-So_Nice__Summer_Samba_’ ‘Bela_Bartok/The_Miraculous_Mandarin_._Music_For_Strings__Percussion_and_Celesta/20sec/03-The_Miraculous_Mandarin___3__2nd_Decoy_Game’ ‘Bela_Bartok/The_Miraculous_Mandarin_._Music_For_Strings__Percussion_and_Celesta/20sec/05-The_Miraculous_Mandarin___5.1__Maestoso___5.2__Allegro’ … CSV

1.0 compared with schreiber2018/cnn (165 differences): ‘Abba/Gold - Greatest Hits/20 sec/16-One Of Us’ ‘Abba/Gold - Greatest Hits/20 sec/17-The Name Of The Game’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/03-The_Tell-tale_Heart’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/04-The_Cask_Of_Amontillado’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/09-The_Fall_Of_The_House_Of_Usher__IV__Pavane’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/01-Tarrega_ Recuerdos de la Alhambra’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/02-Lauro_ Dos Valses Venezolanos - Uno’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/01-Assassin’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/03-Black_White’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/04-Buzzing’ … CSV

1.0 compared with schreiber2018/fcn (151 differences): ‘Abba/Gold - Greatest Hits/20 sec/16-One Of Us’ ‘Abba/Gold - Greatest Hits/20 sec/17-The Name Of The Game’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/04-The_Cask_Of_Amontillado’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/09-The_Fall_Of_The_House_Of_Usher__IV__Pavane’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/01-Tarrega_ Recuerdos de la Alhambra’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/02-Lauro_ Dos Valses Venezolanos - Uno’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/01-Assassin’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/04-Buzzing’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/06-Modern_Apprentice’ … CSV

1.0 compared with schreiber2018/ismir2018 (192 differences): ‘Abba/Gold - Greatest Hits/20 sec/16-One Of Us’ ‘Abba/Gold - Greatest Hits/20 sec/17-The Name Of The Game’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/03-The_Tell-tale_Heart’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/04-The_Cask_Of_Amontillado’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/09-The_Fall_Of_The_House_Of_Usher__IV__Pavane’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/01-Tarrega_ Recuerdos de la Alhambra’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/02-Lauro_ Dos Valses Venezolanos - Uno’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/04-Buzzing’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/06-Modern_Apprentice’ … CSV

1.0 compared with sun2021/default (192 differences): ‘Abba/Gold - Greatest Hits/20 sec/16-One Of Us’ ‘Abba/Gold - Greatest Hits/20 sec/17-The Name Of The Game’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/03-The_Tell-tale_Heart’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/04-The_Cask_Of_Amontillado’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/09-The_Fall_Of_The_House_Of_Usher__IV__Pavane’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/02-Lauro_ Dos Valses Venezolanos - Uno’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/01-Assassin’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/02-Change’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/03-Black_White’ … CSV

1.0 compared with tzanetakis2002/ismir2004_hist (366 differences): ‘Abba/Gold - Greatest Hits/20 sec/01-Dancing Queen’ ‘Abba/Gold - Greatest Hits/20 sec/07-I Have A Dream’ ‘Abba/Gold - Greatest Hits/20 sec/08-The Winner Takes It All’ ‘Abba/Gold - Greatest Hits/20 sec/10-S. O. S’ ‘Abba/Gold - Greatest Hits/20 sec/11-Chiquitita’ ‘Abba/Gold - Greatest Hits/20 sec/12-Fernando’ ‘Abba/Gold - Greatest Hits/20 sec/16-One Of Us’ ‘Abba/Gold - Greatest Hits/20 sec/17-The Name Of The Game’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/02-The_Raven’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/04-The_Cask_Of_Amontillado’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/09-The_Fall_Of_The_House_Of_Usher__IV__Pavane’ … CSV

1.0 compared with tzanetakis2002/ismir2004_mmul (377 differences): ‘Abba/Gold - Greatest Hits/20 sec/07-I Have A Dream’ ‘Abba/Gold - Greatest Hits/20 sec/08-The Winner Takes It All’ ‘Abba/Gold - Greatest Hits/20 sec/10-S. O. S’ ‘Abba/Gold - Greatest Hits/20 sec/11-Chiquitita’ ‘Abba/Gold - Greatest Hits/20 sec/12-Fernando’ ‘Abba/Gold - Greatest Hits/20 sec/16-One Of Us’ ‘Abba/Gold - Greatest Hits/20 sec/17-The Name Of The Game’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/02-The_Raven’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/03-The_Tell-tale_Heart’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/04-The_Cask_Of_Amontillado’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/05-_The_System_Of__Doctor_Tarr_And_Professor_Fether’ … CSV

1.0 compared with tzanetakis2002/ismir2004_msum (338 differences): ‘Abba/Gold - Greatest Hits/20 sec/07-I Have A Dream’ ‘Abba/Gold - Greatest Hits/20 sec/08-The Winner Takes It All’ ‘Abba/Gold - Greatest Hits/20 sec/11-Chiquitita’ ‘Abba/Gold - Greatest Hits/20 sec/12-Fernando’ ‘Abba/Gold - Greatest Hits/20 sec/16-One Of Us’ ‘Abba/Gold - Greatest Hits/20 sec/17-The Name Of The Game’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/02-The_Raven’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/03-The_Tell-tale_Heart’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/04-The_Cask_Of_Amontillado’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/05-_The_System_Of__Doctor_Tarr_And_Professor_Fether’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/09-The_Fall_Of_The_House_Of_Usher__IV__Pavane’ … CSV

1.0 compared with uhle2004/ismir2004 (270 differences): ‘Abba/Gold - Greatest Hits/20 sec/01-Dancing Queen’ ‘Abba/Gold - Greatest Hits/20 sec/08-The Winner Takes It All’ ‘Abba/Gold - Greatest Hits/20 sec/12-Fernando’ ‘Abba/Gold - Greatest Hits/20 sec/16-One Of Us’ ‘Abba/Gold - Greatest Hits/20 sec/17-The Name Of The Game’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/04-The_Cask_Of_Amontillado’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/05-_The_System_Of__Doctor_Tarr_And_Professor_Fether’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/09-The_Fall_Of_The_House_Of_Usher__IV__Pavane’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/01-Tarrega_ Recuerdos de la Alhambra’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ … CSV

1.0 compared with zplane/auftakt_v3 (204 differences): ‘Abba/Gold - Greatest Hits/20 sec/08-The Winner Takes It All’ ‘Abba/Gold - Greatest Hits/20 sec/12-Fernando’ ‘Abba/Gold - Greatest Hits/20 sec/17-The Name Of The Game’ ‘Abba/Gold - Greatest Hits/20 sec/19-Waterloo’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/03-The_Tell-tale_Heart’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/04-The_Cask_Of_Amontillado’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/09-The_Fall_Of_The_House_Of_Usher__IV__Pavane’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/01-Tarrega_ Recuerdos de la Alhambra’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/02-Lauro_ Dos Valses Venezolanos - Uno’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ … CSV

None of the estimators estimated the following 12 items ‘correctly’ using Accuracy1: ‘Asian_Dub_Foundation/R.A.F.I/20sec/13-Real_Areas_For_Investigation’ ‘Bach/20 sec/Bach - Cello suite 2 d - 01 Prelude’ ‘Bernstein_conducts_Stravinsky/20sec/01-Introduction__Part_1-_The_adoration_of_the_earth_’ ‘Bernstein_conducts_Stravinsky/20sec/16-Legerdemain_scene-_Russian_danse’ ‘Bernstein_conducts_Stravinsky/20sec/19-Waltz__ballerina_and_the_moor’ ‘Classic/20 sec/Pergolesi - Stabat Mater 01’ ‘Fado/20 sec/03-AudioTrack 03’ ‘Papakonstantinou/20 sec/02-AudioTrack 02’ ‘Papakonstantinou/20 sec/13-AudioTrack 13’ ‘more greek/03 Virginouda’ ‘more greek/Mpellou - Min klais’ … CSV

Differing Items Accuracy2

Items with different tempo annotations (Accuracy2, 4% tolerance) in different versions:

1.0 compared with alonso2004/ismir2004_corr (196 differences): ‘Abba/Gold - Greatest Hits/20 sec/01-Dancing Queen’ ‘Abba/Gold - Greatest Hits/20 sec/02-Knowing Me, Knowing You’ ‘Abba/Gold - Greatest Hits/20 sec/03-Take A Chance On Me’ ‘Abba/Gold - Greatest Hits/20 sec/08-The Winner Takes It All’ ‘Abba/Gold - Greatest Hits/20 sec/10-S. O. S’ ‘Abba/Gold - Greatest Hits/20 sec/11-Chiquitita’ ‘Abba/Gold - Greatest Hits/20 sec/12-Fernando’ ‘Abba/Gold - Greatest Hits/20 sec/14-Gimme! Gimme! Gimme! (A Man After Midnight)’ ‘Abba/Gold - Greatest Hits/20 sec/15-Does Your Mother Know’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/02-The_Raven’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/05-_The_System_Of__Doctor_Tarr_And_Professor_Fether’ … CSV

1.0 compared with alonso2004/ismir2004_spec (146 differences): ‘Abba/Gold - Greatest Hits/20 sec/01-Dancing Queen’ ‘Abba/Gold - Greatest Hits/20 sec/03-Take A Chance On Me’ ‘Abba/Gold - Greatest Hits/20 sec/08-The Winner Takes It All’ ‘Abba/Gold - Greatest Hits/20 sec/10-S. O. S’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/02-The_Raven’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/04-The_Cask_Of_Amontillado’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/01-Tarrega_ Recuerdos de la Alhambra’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/02-Lauro_ Dos Valses Venezolanos - Uno’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/06-Modern_Apprentice’ … CSV

1.0 compared with boeck2015/tempodetector2016_default (24 differences): ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/09-The_Fall_Of_The_House_Of_Usher__IV__Pavane’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/02-Lauro_ Dos Valses Venezolanos - Uno’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/13-Real_Areas_For_Investigation’ ‘Bach - Walcha/DGG 419 047-2/20sec/04-Praludium u. Fuge f-moll BWV 534 - II_ Fuge’ ‘Bach/20 sec/Bach - Cello suite 2 d - 01 Prelude’ ‘Bela_Bartok/The_Miraculous_Mandarin_._Music_For_Strings__Percussion_and_Celesta/20sec/01-The_Miraculous_Mandarin___1.1__Allegro___1.2__Curtain’ ‘Bela_Bartok/The_Miraculous_Mandarin_._Music_For_Strings__Percussion_and_Celesta/20sec/02-The_Miraculous_Mandarin___2__1st_Decoy_Game’ ‘Bela_Bartok/The_Miraculous_Mandarin_._Music_For_Strings__Percussion_and_Celesta/20sec/03-The_Miraculous_Mandarin___3__2nd_Decoy_Game’ ‘Bela_Bartok/The_Miraculous_Mandarin_._Music_For_Strings__Percussion_and_Celesta/20sec/11-Music_for_Strings__Percussion_and_Celesta__III__Adagio’ … CSV

1.0 compared with davies2009/mirex_qm_tempotracker (84 differences): ‘Abba/Gold - Greatest Hits/20 sec/08-The Winner Takes It All’ ‘Abba/Gold - Greatest Hits/20 sec/11-Chiquitita’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/01-Tarrega_ Recuerdos de la Alhambra’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/01-Assassin’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/04-Buzzing’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/07-Operation_Eagle_Lie’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/12-Culture_Move’ ‘Autechre/Amber/20sec/10-Nil’ ‘Aviador_Dro/Primer_Aliento/20sec/04-Varsovia_en_llamas’ … CSV

1.0 compared with dixon2001/ismir2004_indu (173 differences): ‘Abba/Gold - Greatest Hits/20 sec/08-The Winner Takes It All’ ‘Abba/Gold - Greatest Hits/20 sec/11-Chiquitita’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/02-The_Raven’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/04-The_Cask_Of_Amontillado’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/11-To_One_In_Paradise’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/01-Tarrega_ Recuerdos de la Alhambra’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/13-Real_Areas_For_Investigation’ ‘Autechre/Amber/20sec/02-Montreal’ ‘Autechre/Amber/20sec/06-Piezo’ … CSV

1.0 compared with dixon2001/ismir2004_trac (145 differences): ‘Abba/Gold - Greatest Hits/20 sec/11-Chiquitita’ ‘Abba/Gold - Greatest Hits/20 sec/12-Fernando’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/02-The_Raven’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/04-The_Cask_Of_Amontillado’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/09-The_Fall_Of_The_House_Of_Usher__IV__Pavane’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/01-Tarrega_ Recuerdos de la Alhambra’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/13-Real_Areas_For_Investigation’ ‘Autechre/Amber/20sec/06-Piezo’ ‘Bach - Walcha/DGG 419 047-2/20sec/03-Praludium u. Fuge f-moll BWV 534 - I_ Praludium’ … CSV

1.0 compared with dixon2003/ismir2004_auco (108 differences): ‘Abba/Gold - Greatest Hits/20 sec/01-Dancing Queen’ ‘Abba/Gold - Greatest Hits/20 sec/11-Chiquitita’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/09-The_Fall_Of_The_House_Of_Usher__IV__Pavane’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/01-Tarrega_ Recuerdos de la Alhambra’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/04-Buzzing’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/13-Real_Areas_For_Investigation’ ‘Bach - Walcha/DGG 419 047-2/20sec/02-Toccata u. Fuge d-moll BWV 565 - II_ Fuge’ ‘Bach - Walcha/DGG 419 047-2/20sec/03-Praludium u. Fuge f-moll BWV 534 - I_ Praludium’ ‘Bach - Walcha/DGG 419 047-2/20sec/04-Praludium u. Fuge f-moll BWV 534 - II_ Fuge’ … CSV

1.0 compared with echonest/version_3_2_1 (100 differences): ‘Abba/Gold - Greatest Hits/20 sec/08-The Winner Takes It All’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/04-The_Cask_Of_Amontillado’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/01-Tarrega_ Recuerdos de la Alhambra’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/09-Naxalite’ ‘Autechre/Amber/20sec/10-Nil’ ‘Bach - Walcha/DGG 419 047-2/20sec/03-Praludium u. Fuge f-moll BWV 534 - I_ Praludium’ ‘Bach - Walcha/DGG 419 047-2/20sec/04-Praludium u. Fuge f-moll BWV 534 - II_ Fuge’ ‘Bach - Walcha/DGG 419 047-2/20sec/08-Toccata, Adagio u. Fuge C-dur BWV 564 - II_ Adagio’ ‘Bach - Walcha/DGG 419 047-2/20sec/11-Triosonate Nr. 1 Es-Dur BWV 525 - II_ Adagio’ … CSV

1.0 compared with gkiokas2012/default (43 differences): ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/09-The_Fall_Of_The_House_Of_Usher__IV__Pavane’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/01-Tarrega_ Recuerdos de la Alhambra’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/06-Modern_Apprentice’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/13-Real_Areas_For_Investigation’ ‘Autechre/Amber/20sec/06-Piezo’ ‘Bach - Walcha/DGG 419 047-2/20sec/04-Praludium u. Fuge f-moll BWV 534 - II_ Fuge’ ‘Bach - Walcha/DGG 419 047-2/20sec/08-Toccata, Adagio u. Fuge C-dur BWV 564 - II_ Adagio’ ‘Bach/20 sec/Bach - BWV 639 - Ich ruf’ zu dir, Herr Jesu Christ’ ‘Bach/20 sec/Bach - Cello suite 2 d - 01 Prelude’ … CSV

1.0 compared with klapuri2006/ismir2004 (42 differences): ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/09-The_Fall_Of_The_House_Of_Usher__IV__Pavane’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/01-Tarrega_ Recuerdos de la Alhambra’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/07-Operation_Eagle_Lie’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/12-Culture_Move’ ‘Aviador_Dro/Primer_Aliento/20sec/02-Nuclear__si__por_supuesto_’ ‘Bach - Walcha/DGG 419 047-2/20sec/08-Toccata, Adagio u. Fuge C-dur BWV 564 - II_ Adagio’ ‘Bach/20 sec/Bach - BWV 639 - Ich ruf’ zu dir, Herr Jesu Christ’ ‘Bela_Bartok/The_Miraculous_Mandarin_._Music_For_Strings__Percussion_and_Celesta/20sec/02-The_Miraculous_Mandarin___2__1st_Decoy_Game’ ‘Bela_Bartok/The_Miraculous_Mandarin_._Music_For_Strings__Percussion_and_Celesta/20sec/03-The_Miraculous_Mandarin___3__2nd_Decoy_Game’ ‘Bela_Bartok/The_Miraculous_Mandarin_._Music_For_Strings__Percussion_and_Celesta/20sec/06-The_Miraculous_Mandarin___6.1__Sempre_vivo___6.2__Adagio’ … CSV

1.0 compared with klapuri2006/percival2014 (50 differences): ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/09-The_Fall_Of_The_House_Of_Usher__IV__Pavane’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/01-Tarrega_ Recuerdos de la Alhambra’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/07-Operation_Eagle_Lie’ ‘Bach/20 sec/Bach - BWV 639 - Ich ruf’ zu dir, Herr Jesu Christ’ ‘Bach/20 sec/Bach - Cello suite 2 d - 01 Prelude’ ‘Bela_Bartok/The_Miraculous_Mandarin_._Music_For_Strings__Percussion_and_Celesta/20sec/02-The_Miraculous_Mandarin___2__1st_Decoy_Game’ ‘Bela_Bartok/The_Miraculous_Mandarin_._Music_For_Strings__Percussion_and_Celesta/20sec/03-The_Miraculous_Mandarin___3__2nd_Decoy_Game’ ‘Bela_Bartok/The_Miraculous_Mandarin_._Music_For_Strings__Percussion_and_Celesta/20sec/06-The_Miraculous_Mandarin___6.1__Sempre_vivo___6.2__Adagio’ ‘Bela_Bartok/The_Miraculous_Mandarin_._Music_For_Strings__Percussion_and_Celesta/20sec/07-The_Miraculous_Mandarin___7.1__Agitato___7.2__Molto_moderato’ … CSV

1.0 compared with oliveira2010/ibt (108 differences): ‘Abba/Gold - Greatest Hits/20 sec/08-The Winner Takes It All’ ‘Abba/Gold - Greatest Hits/20 sec/11-Chiquitita’ ‘Abba/Gold - Greatest Hits/20 sec/12-Fernando’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/01-Tarrega_ Recuerdos de la Alhambra’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/02-Lauro_ Dos Valses Venezolanos - Uno’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/13-Real_Areas_For_Investigation’ ‘Autechre/Amber/20sec/10-Nil’ ‘Bach - Walcha/DGG 419 047-2/20sec/04-Praludium u. Fuge f-moll BWV 534 - II_ Fuge’ ‘Bach - Walcha/DGG 419 047-2/20sec/11-Triosonate Nr. 1 Es-Dur BWV 525 - II_ Adagio’ … CSV

1.0 compared with percival2014/stem (65 differences): ‘Abba/Gold - Greatest Hits/20 sec/11-Chiquitita’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/01-Tarrega_ Recuerdos de la Alhambra’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/02-Lauro_ Dos Valses Venezolanos - Uno’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/07-Operation_Eagle_Lie’ ‘Autechre/Amber/20sec/10-Nil’ ‘Bach - Walcha/DGG 419 047-2/20sec/08-Toccata, Adagio u. Fuge C-dur BWV 564 - II_ Adagio’ ‘Bach/20 sec/Bach - Cello suite 2 d - 01 Prelude’ ‘Bebel_Gilberto/Tanto_Tempo/20sec/06-Alguem’ ‘Bela_Bartok/The_Miraculous_Mandarin_._Music_For_Strings__Percussion_and_Celesta/20sec/01-The_Miraculous_Mandarin___1.1__Allegro___1.2__Curtain’ … CSV

1.0 compared with scheirer1998/ismir2004 (142 differences): ‘Abba/Gold - Greatest Hits/20 sec/04-Mamma Mia’ ‘Abba/Gold - Greatest Hits/20 sec/08-The Winner Takes It All’ ‘Abba/Gold - Greatest Hits/20 sec/09-Money, Money, Money’ ‘Abba/Gold - Greatest Hits/20 sec/19-Waterloo’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/02-The_Raven’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/05-_The_System_Of__Doctor_Tarr_And_Professor_Fether’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/02-Lauro_ Dos Valses Venezolanos - Uno’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/01-Assassin’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/12-Culture_Move’ ‘Autechre/Amber/20sec/01-Foil’ … CSV

1.0 compared with scheirer1998/percival2014 (166 differences): ‘Abba/Gold - Greatest Hits/20 sec/11-Chiquitita’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/02-The_Raven’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/05-_The_System_Of__Doctor_Tarr_And_Professor_Fether’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/02-Lauro_ Dos Valses Venezolanos - Uno’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/03-Black_White’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/05-Free_Satpal_Ram’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/09-Naxalite’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/12-Culture_Move’ ‘Autechre/Amber/20sec/01-Foil’ … CSV

1.0 compared with schreiber2014/default (38 differences): ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/09-The_Fall_Of_The_House_Of_Usher__IV__Pavane’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/01-Tarrega_ Recuerdos de la Alhambra’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/02-Lauro_ Dos Valses Venezolanos - Uno’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/04-Buzzing’ ‘Bach - Walcha/DGG 419 047-2/20sec/04-Praludium u. Fuge f-moll BWV 534 - II_ Fuge’ ‘Bach/20 sec/Bach - Cello suite 2 d - 01 Prelude’ ‘Bela_Bartok/The_Miraculous_Mandarin_._Music_For_Strings__Percussion_and_Celesta/20sec/01-The_Miraculous_Mandarin___1.1__Allegro___1.2__Curtain’ ‘Bela_Bartok/The_Miraculous_Mandarin_._Music_For_Strings__Percussion_and_Celesta/20sec/02-The_Miraculous_Mandarin___2__1st_Decoy_Game’ ‘Bela_Bartok/The_Miraculous_Mandarin_._Music_For_Strings__Percussion_and_Celesta/20sec/03-The_Miraculous_Mandarin___3__2nd_Decoy_Game’ … CSV

1.0 compared with schreiber2017/ismir2017 (40 differences): ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/09-The_Fall_Of_The_House_Of_Usher__IV__Pavane’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/02-Lauro_ Dos Valses Venezolanos - Uno’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/04-Buzzing’ ‘Bach - Walcha/DGG 419 047-2/20sec/04-Praludium u. Fuge f-moll BWV 534 - II_ Fuge’ ‘Bach/20 sec/Bach - Cello suite 2 d - 01 Prelude’ ‘Bela_Bartok/The_Miraculous_Mandarin_._Music_For_Strings__Percussion_and_Celesta/20sec/03-The_Miraculous_Mandarin___3__2nd_Decoy_Game’ ‘Bela_Bartok/The_Miraculous_Mandarin_._Music_For_Strings__Percussion_and_Celesta/20sec/06-The_Miraculous_Mandarin___6.1__Sempre_vivo___6.2__Adagio’ ‘Bela_Bartok/The_Miraculous_Mandarin_._Music_For_Strings__Percussion_and_Celesta/20sec/09-Music_for_Strings__Percussion_and_Celesta__I__Andante_tranquillo’ ‘Bernstein_conducts_Stravinsky/20sec/01-Introduction__Part_1-_The_adoration_of_the_earth_’ … CSV

1.0 compared with schreiber2017/mirex2017 (24 differences): ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ ‘Bach - Walcha/DGG 419 047-2/20sec/04-Praludium u. Fuge f-moll BWV 534 - II_ Fuge’ ‘Bach/20 sec/Bach - Cello suite 2 d - 01 Prelude’ ‘Bela_Bartok/The_Miraculous_Mandarin_._Music_For_Strings__Percussion_and_Celesta/20sec/03-The_Miraculous_Mandarin___3__2nd_Decoy_Game’ ‘Bela_Bartok/The_Miraculous_Mandarin_._Music_For_Strings__Percussion_and_Celesta/20sec/06-The_Miraculous_Mandarin___6.1__Sempre_vivo___6.2__Adagio’ ‘Bela_Bartok/The_Miraculous_Mandarin_._Music_For_Strings__Percussion_and_Celesta/20sec/09-Music_for_Strings__Percussion_and_Celesta__I__Andante_tranquillo’ ‘Bernstein_conducts_Stravinsky/20sec/01-Introduction__Part_1-_The_adoration_of_the_earth_’ ‘Bernstein_conducts_Stravinsky/20sec/02-The_augurs_of_spring’ ‘Bernstein_conducts_Stravinsky/20sec/12-Summoning_of_the_ancestors’ ‘Bernstein_conducts_Stravinsky/20sec/16-Legerdemain_scene-_Russian_danse’ … CSV

1.0 compared with schreiber2018/cnn (36 differences): ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/04-The_Cask_Of_Amontillado’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/09-The_Fall_Of_The_House_Of_Usher__IV__Pavane’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/01-Tarrega_ Recuerdos de la Alhambra’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Bach - Walcha/DGG 419 047-2/20sec/04-Praludium u. Fuge f-moll BWV 534 - II_ Fuge’ ‘Bach - Walcha/DGG 419 047-2/20sec/08-Toccata, Adagio u. Fuge C-dur BWV 564 - II_ Adagio’ ‘Bach/20 sec/Bach - BWV 639 - Ich ruf’ zu dir, Herr Jesu Christ’ ‘Bach/20 sec/Bach - Cello suite 1 G - 01 Prelude’ ‘Bach/20 sec/Bach - Cello suite 2 d - 01 Prelude’ ‘Bela_Bartok/The_Miraculous_Mandarin_._Music_For_Strings__Percussion_and_Celesta/20sec/02-The_Miraculous_Mandarin___2__1st_Decoy_Game’ ‘Bela_Bartok/The_Miraculous_Mandarin_._Music_For_Strings__Percussion_and_Celesta/20sec/03-The_Miraculous_Mandarin___3__2nd_Decoy_Game’ … CSV

1.0 compared with schreiber2018/fcn (30 differences): ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/04-The_Cask_Of_Amontillado’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/09-The_Fall_Of_The_House_Of_Usher__IV__Pavane’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/01-Tarrega_ Recuerdos de la Alhambra’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ ‘Bach - Walcha/DGG 419 047-2/20sec/04-Praludium u. Fuge f-moll BWV 534 - II_ Fuge’ ‘Bach/20 sec/Bach - BWV 639 - Ich ruf’ zu dir, Herr Jesu Christ’ ‘Bach/20 sec/Bach - Cello suite 2 d - 01 Prelude’ ‘Bela_Bartok/The_Miraculous_Mandarin_._Music_For_Strings__Percussion_and_Celesta/20sec/01-The_Miraculous_Mandarin___1.1__Allegro___1.2__Curtain’ ‘Bela_Bartok/The_Miraculous_Mandarin_._Music_For_Strings__Percussion_and_Celesta/20sec/02-The_Miraculous_Mandarin___2__1st_Decoy_Game’ ‘Bela_Bartok/The_Miraculous_Mandarin_._Music_For_Strings__Percussion_and_Celesta/20sec/03-The_Miraculous_Mandarin___3__2nd_Decoy_Game’ … CSV

1.0 compared with schreiber2018/ismir2018 (38 differences): ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/09-The_Fall_Of_The_House_Of_Usher__IV__Pavane’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/01-Tarrega_ Recuerdos de la Alhambra’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ ‘Bach - Walcha/DGG 419 047-2/20sec/04-Praludium u. Fuge f-moll BWV 534 - II_ Fuge’ ‘Bach - Walcha/DGG 419 047-2/20sec/08-Toccata, Adagio u. Fuge C-dur BWV 564 - II_ Adagio’ ‘Bach/20 sec/Bach - BWV 639 - Ich ruf’ zu dir, Herr Jesu Christ’ ‘Bach/20 sec/Bach - Cello suite 1 G - 01 Prelude’ ‘Bach/20 sec/Bach - Cello suite 2 d - 01 Prelude’ ‘Bela_Bartok/The_Miraculous_Mandarin_._Music_For_Strings__Percussion_and_Celesta/20sec/01-The_Miraculous_Mandarin___1.1__Allegro___1.2__Curtain’ ‘Bela_Bartok/The_Miraculous_Mandarin_._Music_For_Strings__Percussion_and_Celesta/20sec/02-The_Miraculous_Mandarin___2__1st_Decoy_Game’ … CSV

1.0 compared with sun2021/default (44 differences): ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/09-The_Fall_Of_The_House_Of_Usher__IV__Pavane’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/02-Lauro_ Dos Valses Venezolanos - Uno’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/13-Real_Areas_For_Investigation’ ‘Autechre/Amber/20sec/11-Teartear’ ‘Bach/20 sec/Bach - BWV 639 - Ich ruf’ zu dir, Herr Jesu Christ’ ‘Bach/20 sec/Bach - Cello suite 1 G - 01 Prelude’ ‘Bach/20 sec/Bach - Cello suite 1 G - 03 Courante’ ‘Bach/20 sec/Bach - Cello suite 2 d - 01 Prelude’ ‘Bela_Bartok/The_Miraculous_Mandarin_._Music_For_Strings__Percussion_and_Celesta/20sec/02-The_Miraculous_Mandarin___2__1st_Decoy_Game’ … CSV

1.0 compared with tzanetakis2002/ismir2004_hist (243 differences): ‘Abba/Gold - Greatest Hits/20 sec/07-I Have A Dream’ ‘Abba/Gold - Greatest Hits/20 sec/11-Chiquitita’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/04-The_Cask_Of_Amontillado’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/11-To_One_In_Paradise’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/01-Tarrega_ Recuerdos de la Alhambra’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/02-Lauro_ Dos Valses Venezolanos - Uno’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/01-Assassin’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/02-Change’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/04-Buzzing’ … CSV

1.0 compared with tzanetakis2002/ismir2004_mmul (273 differences): ‘Abba/Gold - Greatest Hits/20 sec/08-The Winner Takes It All’ ‘Abba/Gold - Greatest Hits/20 sec/10-S. O. S’ ‘Abba/Gold - Greatest Hits/20 sec/11-Chiquitita’ ‘Abba/Gold - Greatest Hits/20 sec/12-Fernando’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/02-The_Raven’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/04-The_Cask_Of_Amontillado’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/05-_The_System_Of__Doctor_Tarr_And_Professor_Fether’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/07-The_Fall_Of_The_House_Of_Usher__II__Arrival’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/09-The_Fall_Of_The_House_Of_Usher__IV__Pavane’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/11-To_One_In_Paradise’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/01-Tarrega_ Recuerdos de la Alhambra’ … CSV

1.0 compared with tzanetakis2002/ismir2004_msum (221 differences): ‘Abba/Gold - Greatest Hits/20 sec/07-I Have A Dream’ ‘Abba/Gold - Greatest Hits/20 sec/08-The Winner Takes It All’ ‘Abba/Gold - Greatest Hits/20 sec/11-Chiquitita’ ‘Abba/Gold - Greatest Hits/20 sec/12-Fernando’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/02-The_Raven’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/04-The_Cask_Of_Amontillado’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/05-_The_System_Of__Doctor_Tarr_And_Professor_Fether’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/09-The_Fall_Of_The_House_Of_Usher__IV__Pavane’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/01-Tarrega_ Recuerdos de la Alhambra’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ … CSV

1.0 compared with uhle2004/ismir2004 (130 differences): ‘Abba/Gold - Greatest Hits/20 sec/01-Dancing Queen’ ‘Abba/Gold - Greatest Hits/20 sec/08-The Winner Takes It All’ ‘Abba/Gold - Greatest Hits/20 sec/12-Fernando’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/05-_The_System_Of__Doctor_Tarr_And_Professor_Fether’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/09-The_Fall_Of_The_House_Of_Usher__IV__Pavane’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/01-Tarrega_ Recuerdos de la Alhambra’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/12-Culture_Move’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/13-Real_Areas_For_Investigation’ ‘Autechre/Amber/20sec/01-Foil’ … CSV

1.0 compared with zplane/auftakt_v3 (82 differences): ‘Abba/Gold - Greatest Hits/20 sec/08-The Winner Takes It All’ ‘Abba/Gold - Greatest Hits/20 sec/12-Fernando’ ‘Alan_Parsons_Project/Tales_Of_Mystery_And_Imagination/20sec/09-The_Fall_Of_The_House_Of_Usher__IV__Pavane’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/01-Tarrega_ Recuerdos de la Alhambra’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/02-Lauro_ Dos Valses Venezolanos - Uno’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/04-Sojo_ Guaso’ ‘Alirio Diaz/The Art of Spanish Guitar/20 sec/05-Sojo_ Cancion’ ‘Asian_Dub_Foundation/R.A.F.I/20sec/09-Naxalite’ ‘Autechre/Amber/20sec/02-Montreal’ ‘Autechre/Amber/20sec/10-Nil’ ‘Bach - Walcha/DGG 419 047-2/20sec/04-Praludium u. Fuge f-moll BWV 534 - II_ Fuge’ … CSV

None of the estimators estimated the following item ‘correctly’ using Accuracy2: ‘Bernstein_conducts_Stravinsky/20sec/16-Legerdemain_scene-_Russian_danse’ CSV

Significance of Differences

Estimator alonso2004/ismir2004_corr alonso2004/ismir2004_spec boeck2015/tempodetector2016_default davies2009/mirex_qm_tempotracker dixon2001/ismir2004_indu dixon2001/ismir2004_trac dixon2003/ismir2004_auco echonest/version_3_2_1 gkiokas2012/default klapuri2006/ismir2004 klapuri2006/percival2014 oliveira2010/ibt percival2014/stem scheirer1998/ismir2004 scheirer1998/percival2014 schreiber2014/default schreiber2017/ismir2017 schreiber2017/mirex2017 schreiber2018/cnn schreiber2018/fcn schreiber2018/ismir2018 sun2021/default tzanetakis2002/ismir2004_hist tzanetakis2002/ismir2004_mmul tzanetakis2002/ismir2004_msum uhle2004/ismir2004 zplane/auftakt_v3
alonso2004/ismir2004_corr 1.0000 0.0000 0.0000 0.0000 0.0587 0.0932 0.0073 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.4469 0.0875 0.1679 0.0000 0.0000
alonso2004/ismir2004_spec 0.0000 1.0000 0.0000 0.0187 0.0013 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0013 0.0000 0.9366 0.0690 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0005 0.1105 0.0000
boeck2015/tempodetector2016_default 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.6342 0.6239 0.1983 0.3099 0.0003 0.0240 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.1509 0.1719 0.0000 0.0000 0.0000 0.0000 0.8423
davies2009/mirex_qm_tempotracker 0.0000 0.0187 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.1366 0.0000 0.0225 0.6658 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.4214 0.0000
dixon2001/ismir2004_indu 0.0587 0.0013 0.0000 0.0000 1.0000 0.0004 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0005 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0063 0.0001 0.6769 0.0000 0.0000
dixon2001/ismir2004_trac 0.0932 0.0000 0.0000 0.0000 0.0004 1.0000 0.2410 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.4743 0.9233 0.0017 0.0000 0.0000
dixon2003/ismir2004_auco 0.0073 0.0000 0.0000 0.0000 0.0000 0.2410 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0778 0.4075 0.0001 0.0000 0.0000
echonest/version_3_2_1 0.0000 0.0000 0.6342 0.0000 0.0000 0.0000 0.0000 1.0000 1.0000 0.6790 0.8039 0.0001 0.1537 0.0000 0.0000 0.0000 0.0001 0.0000 0.0050 0.0001 0.5676 0.5676 0.0000 0.0000 0.0000 0.0000 0.7984
gkiokas2012/default 0.0000 0.0000 0.6239 0.0000 0.0000 0.0000 0.0000 1.0000 1.0000 0.5619 0.7091 0.0001 0.1046 0.0000 0.0000 0.0000 0.0001 0.0000 0.0009 0.0000 0.4633 0.4633 0.0000 0.0000 0.0000 0.0000 0.8508
klapuri2006/ismir2004 0.0000 0.0000 0.1983 0.0000 0.0000 0.0000 0.0000 0.6790 0.5619 1.0000 0.7905 0.0000 0.2070 0.0000 0.0000 0.0000 0.0002 0.0000 0.0037 0.0002 0.9049 0.9226 0.0000 0.0000 0.0000 0.0000 0.3318
klapuri2006/percival2014 0.0000 0.0000 0.3099 0.0000 0.0000 0.0000 0.0000 0.8039 0.7091 0.7905 1.0000 0.0000 0.1307 0.0000 0.0000 0.0000 0.0001 0.0000 0.0021 0.0002 0.7237 0.7768 0.0000 0.0000 0.0000 0.0000 0.4504
oliveira2010/ibt 0.0000 0.0013 0.0003 0.1366 0.0000 0.0000 0.0000 0.0001 0.0001 0.0000 0.0000 1.0000 0.0000 0.0017 0.1350 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0395 0.0000
percival2014/stem 0.0000 0.0000 0.0240 0.0000 0.0000 0.0000 0.0000 0.1537 0.1046 0.2070 0.1307 0.0000 1.0000 0.0000 0.0000 0.0000 0.0059 0.0000 0.1491 0.0140 0.3583 0.4113 0.0000 0.0000 0.0000 0.0000 0.0321
scheirer1998/ismir2004 0.0000 0.9366 0.0000 0.0225 0.0005 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0017 0.0000 1.0000 0.0395 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0003 0.1534 0.0000
scheirer1998/percival2014 0.0000 0.0690 0.0000 0.6658 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.1350 0.0000 0.0395 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.8633 0.0000
schreiber2014/default 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0002 0.0000 0.0000 0.0116 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2017/ismir2017 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0001 0.0001 0.0002 0.0001 0.0000 0.0059 0.0000 0.0000 0.0002 1.0000 0.0000 0.3291 0.8453 0.0004 0.0007 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2017/mirex2017 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2018/cnn 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0050 0.0009 0.0037 0.0021 0.0000 0.1491 0.0000 0.0000 0.0000 0.3291 0.0000 1.0000 0.1753 0.0018 0.0076 0.0000 0.0000 0.0000 0.0000 0.0003
schreiber2018/fcn 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0001 0.0000 0.0002 0.0002 0.0000 0.0140 0.0000 0.0000 0.0116 0.8453 0.0000 0.1753 1.0000 0.0002 0.0001 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2018/ismir2018 0.0000 0.0000 0.1509 0.0000 0.0000 0.0000 0.0000 0.5676 0.4633 0.9049 0.7237 0.0000 0.3583 0.0000 0.0000 0.0000 0.0004 0.0000 0.0018 0.0002 1.0000 0.9170 0.0000 0.0000 0.0000 0.0000 0.2616
sun2021/default 0.0000 0.0000 0.1719 0.0000 0.0000 0.0000 0.0000 0.5676 0.4633 0.9226 0.7768 0.0000 0.4113 0.0000 0.0000 0.0000 0.0007 0.0000 0.0076 0.0001 0.9170 1.0000 0.0000 0.0000 0.0000 0.0000 0.3153
tzanetakis2002/ismir2004_hist 0.4469 0.0000 0.0000 0.0000 0.0063 0.4743 0.0778 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.2665 0.0005 0.0000 0.0000
tzanetakis2002/ismir2004_mmul 0.0875 0.0000 0.0000 0.0000 0.0001 0.9233 0.4075 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.2665 1.0000 0.0000 0.0000 0.0000
tzanetakis2002/ismir2004_msum 0.1679 0.0005 0.0000 0.0000 0.6769 0.0017 0.0001 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0003 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0005 0.0000 1.0000 0.0000 0.0000
uhle2004/ismir2004 0.0000 0.1105 0.0000 0.4214 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0395 0.0000 0.1534 0.8633 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000
zplane/auftakt_v3 0.0000 0.0000 0.8423 0.0000 0.0000 0.0000 0.0000 0.7984 0.8508 0.3318 0.4504 0.0000 0.0321 0.0000 0.0000 0.0000 0.0000 0.0000 0.0003 0.0000 0.2616 0.3153 0.0000 0.0000 0.0000 0.0000 1.0000

Table 4: McNemar p-values, using reference annotations 1.0 as groundtruth with Accuracy1 [Gouyon2006]. H0: both estimators disagree with the groundtruth to the same amount. If p<=ɑ, reject H0, i.e. we have a significant difference in the disagreement with the groundtruth. In the table, p-values<0.05 are set in bold.

CSV JSON LATEX PICKLE

Estimator alonso2004/ismir2004_corr alonso2004/ismir2004_spec boeck2015/tempodetector2016_default davies2009/mirex_qm_tempotracker dixon2001/ismir2004_indu dixon2001/ismir2004_trac dixon2003/ismir2004_auco echonest/version_3_2_1 gkiokas2012/default klapuri2006/ismir2004 klapuri2006/percival2014 oliveira2010/ibt percival2014/stem scheirer1998/ismir2004 scheirer1998/percival2014 schreiber2014/default schreiber2017/ismir2017 schreiber2017/mirex2017 schreiber2018/cnn schreiber2018/fcn schreiber2018/ismir2018 sun2021/default tzanetakis2002/ismir2004_hist tzanetakis2002/ismir2004_mmul tzanetakis2002/ismir2004_msum uhle2004/ismir2004 zplane/auftakt_v3
alonso2004/ismir2004_corr 1.0000 0.0000 0.0000 0.0000 0.0963 0.0001 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0001 0.0316 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0008 0.0000 0.0857 0.0000 0.0000
alonso2004/ismir2004_spec 0.0000 1.0000 0.0000 0.0000 0.0332 1.0000 0.0013 0.0000 0.0000 0.0000 0.0000 0.0008 0.0000 0.8102 0.1258 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.1814 0.0000
boeck2015/tempodetector2016_default 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0005 0.0064 0.0001 0.0000 0.0000 0.0000 0.0000 0.0125 0.0070 1.0000 0.0501 0.3449 0.0201 0.0008 0.0000 0.0000 0.0000 0.0000 0.0000
davies2009/mirex_qm_tempotracker 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0255 0.1258 0.0000 0.0000 0.0000 0.0165 0.0295 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.9131
dixon2001/ismir2004_indu 0.0963 0.0332 0.0000 0.0000 1.0000 0.0129 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0127 0.6299 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0004 0.0004 0.0000
dixon2001/ismir2004_trac 0.0001 1.0000 0.0000 0.0000 0.0129 1.0000 0.0006 0.0001 0.0000 0.0000 0.0000 0.0013 0.0000 0.8707 0.1172 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.2105 0.0000
dixon2003/ismir2004_auco 0.0000 0.0013 0.0000 0.0255 0.0000 0.0006 1.0000 0.5228 0.0000 0.0000 0.0000 0.9284 0.0000 0.0060 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0492 0.0142
echonest/version_3_2_1 0.0000 0.0000 0.0000 0.1258 0.0000 0.0001 0.5228 1.0000 0.0000 0.0000 0.0000 0.4839 0.0004 0.0004 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0061 0.0700
gkiokas2012/default 0.0000 0.0000 0.0005 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 1.0000 0.3604 0.0000 0.0054 0.0000 0.0000 0.5327 0.7608 0.0013 0.2810 0.0192 0.4731 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000
klapuri2006/ismir2004 0.0000 0.0000 0.0064 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 1.0000 0.1516 0.0000 0.0038 0.0000 0.0000 0.6440 0.8830 0.0064 0.3915 0.0576 0.6440 0.8804 0.0000 0.0000 0.0000 0.0000 0.0000
klapuri2006/percival2014 0.0000 0.0000 0.0001 0.0000 0.0000 0.0000 0.0000 0.0000 0.3604 0.1516 1.0000 0.0000 0.0627 0.0000 0.0000 0.1038 0.2026 0.0001 0.0488 0.0029 0.0884 0.4408 0.0000 0.0000 0.0000 0.0000 0.0001
oliveira2010/ibt 0.0000 0.0008 0.0000 0.0165 0.0000 0.0013 0.9284 0.4839 0.0000 0.0000 0.0000 1.0000 0.0000 0.0060 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0433 0.0046
percival2014/stem 0.0000 0.0000 0.0000 0.0295 0.0000 0.0000 0.0000 0.0004 0.0054 0.0038 0.0627 0.0000 1.0000 0.0000 0.0000 0.0004 0.0013 0.0000 0.0004 0.0000 0.0003 0.0111 0.0000 0.0000 0.0000 0.0000 0.0498
scheirer1998/ismir2004 0.0001 0.8102 0.0000 0.0000 0.0127 0.8707 0.0060 0.0004 0.0000 0.0000 0.0000 0.0060 0.0000 1.0000 0.0283 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.3560 0.0000
scheirer1998/percival2014 0.0316 0.1258 0.0000 0.0000 0.6299 0.1172 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0283 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0002 0.0045 0.0000
schreiber2014/default 0.0000 0.0000 0.0125 0.0000 0.0000 0.0000 0.0000 0.0000 0.5327 0.6440 0.1038 0.0000 0.0004 0.0000 0.0000 1.0000 0.8145 0.0125 0.8601 0.2153 1.0000 0.4177 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2017/ismir2017 0.0000 0.0000 0.0070 0.0000 0.0000 0.0000 0.0000 0.0000 0.7608 0.8830 0.2026 0.0000 0.0013 0.0000 0.0000 0.8145 1.0000 0.0001 0.6516 0.1641 0.8776 0.6358 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2017/mirex2017 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0013 0.0064 0.0001 0.0000 0.0000 0.0000 0.0000 0.0125 0.0001 1.0000 0.0501 0.3771 0.0243 0.0017 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2018/cnn 0.0000 0.0000 0.0501 0.0000 0.0000 0.0000 0.0000 0.0000 0.2810 0.3915 0.0488 0.0000 0.0004 0.0000 0.0000 0.8601 0.6516 0.0501 1.0000 0.3616 0.8388 0.2682 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2018/fcn 0.0000 0.0000 0.3449 0.0000 0.0000 0.0000 0.0000 0.0000 0.0192 0.0576 0.0029 0.0000 0.0000 0.0000 0.0000 0.2153 0.1641 0.3771 0.3616 1.0000 0.1849 0.0385 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2018/ismir2018 0.0000 0.0000 0.0201 0.0000 0.0000 0.0000 0.0000 0.0000 0.4731 0.6440 0.0884 0.0000 0.0003 0.0000 0.0000 1.0000 0.8776 0.0243 0.8388 0.1849 1.0000 0.4177 0.0000 0.0000 0.0000 0.0000 0.0000
sun2021/default 0.0000 0.0000 0.0008 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.8804 0.4408 0.0000 0.0111 0.0000 0.0000 0.4177 0.6358 0.0017 0.2682 0.0385 0.4177 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000
tzanetakis2002/ismir2004_hist 0.0008 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0316 0.0717 0.0000 0.0000
tzanetakis2002/ismir2004_mmul 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0316 1.0000 0.0001 0.0000 0.0000
tzanetakis2002/ismir2004_msum 0.0857 0.0000 0.0000 0.0000 0.0004 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0002 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0717 0.0001 1.0000 0.0000 0.0000
uhle2004/ismir2004 0.0000 0.1814 0.0000 0.0000 0.0004 0.2105 0.0492 0.0061 0.0000 0.0000 0.0000 0.0433 0.0000 0.3560 0.0045 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000
zplane/auftakt_v3 0.0000 0.0000 0.0000 0.9131 0.0000 0.0000 0.0142 0.0700 0.0000 0.0000 0.0001 0.0046 0.0498 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000

Table 5: McNemar p-values, using reference annotations 1.0 as groundtruth with Accuracy2 [Gouyon2006]. H0: both estimators disagree with the groundtruth to the same amount. If p<=ɑ, reject H0, i.e. we have a significant difference in the disagreement with the groundtruth. In the table, p-values<0.05 are set in bold.

CSV JSON LATEX PICKLE

Accuracy1 on Tempo-Subsets

How well does an estimator perform, when only taking a subset of the reference annotations into account? The graphs show mean Accuracy1 for reference subsets with tempi in [T-10,T+10] BPM. Note that the graphs do not show confidence intervals and that some values may be based on very few estimates.

Accuracy1 on Tempo-Subsets for 1.0

Figure 5: Mean Accuracy1 for estimates compared to version 1.0 for tempo intervals around T.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy2 on Tempo-Subsets

How well does an estimator perform, when only taking a subset of the reference annotations into account? The graphs show mean Accuracy2 for reference subsets with tempi in [T-10,T+10] BPM. Note that the graphs do not show confidence intervals and that some values may be based on very few estimates.

Accuracy2 on Tempo-Subsets for 1.0

Figure 6: Mean Accuracy2 for estimates compared to version 1.0 for tempo intervals around T.

CSV JSON LATEX PICKLE SVG PDF PNG

Estimated Accuracy1 for Tempo

When fitting a generalized additive model (GAM) to Accuracy1-values and a ground truth, what Accuracy1 can we expect with confidence?

Estimated Accuracy1 for Tempo for 1.0

Predictions of GAMs trained on Accuracy1 for estimates for reference 1.0.

Figure 7: Accuracy1 predictions of a generalized additive model (GAM) fit to Accuracy1 results for 1.0. The 95% confidence interval around the prediction is shaded in gray.

CSV JSON LATEX PICKLE SVG PDF PNG

Estimated Accuracy2 for Tempo

When fitting a generalized additive model (GAM) to Accuracy2-values and a ground truth, what Accuracy2 can we expect with confidence?

Estimated Accuracy2 for Tempo for 1.0

Predictions of GAMs trained on Accuracy2 for estimates for reference 1.0.

Figure 8: Accuracy2 predictions of a generalized additive model (GAM) fit to Accuracy2 results for 1.0. The 95% confidence interval around the prediction is shaded in gray.

CSV JSON LATEX PICKLE SVG PDF PNG

OE1 and OE2

OE1 is defined as octave error between an estimate E and a reference value R.This means that the most common errors—by a factor of 2 or ½—have the same magnitude, namely 1: OE2(E) = log2(E/R).

OE2 is the signed OE1 corresponding to the minimum absolute OE1 allowing the octaveerrors 2, 3, 1/2, and 1/3: OE2(E) = arg minx(|x|) with x ∈ {OE1(E), OE1(2E), OE1(3E), OE1(½E), OE1(⅓E)}

Mean OE1/OE2 Results for 1.0

Estimator OE1_MEAN OE1_STDEV OE2_MEAN OE2_STDEV
schreiber2017/mirex2017 0.0229 0.2842 -0.0030 0.0519
schreiber2014/default 0.1444 0.4590 -0.0130 0.0944
schreiber2017/ismir2017 0.1603 0.5139 -0.0128 0.0918
echonest/version_3_2_1 0.1421 0.5146 0.0106 0.1439
schreiber2018/cnn 0.3019 0.5156 -0.0036 0.0771
sun2021/default 0.3070 0.5233 -0.0124 0.0925
percival2014/stem 0.2066 0.5242 -0.0049 0.1192
klapuri2006/percival2014 0.3460 0.5242 -0.0028 0.1053
oliveira2010/ibt 0.4071 0.5251 0.0019 0.1532
zplane/auftakt_v3 0.3164 0.5254 -0.0152 0.1397
schreiber2018/fcn 0.2372 0.5254 -0.0046 0.0772
klapuri2006/ismir2004 0.3524 0.5275 -0.0051 0.1065
schreiber2018/ismir2018 0.3522 0.5302 -0.0097 0.0982
boeck2015/tempodetector2016_default 0.4316 0.5541 0.0016 0.0812
dixon2003/ismir2004_auco 0.8706 0.5608 0.0383 0.1690
gkiokas2012/default 0.3470 0.5687 0.0064 0.0942
scheirer1998/percival2014 0.2248 0.5798 0.0078 0.1920
dixon2001/ismir2004_trac 0.8089 0.5802 0.0373 0.1747
davies2009/mirex_qm_tempotracker 0.5224 0.5838 0.0462 0.1463
scheirer1998/ismir2004 0.2286 0.6392 0.0352 0.1916
alonso2004/ismir2004_spec 0.1958 0.6404 0.0170 0.1796
alonso2004/ismir2004_corr 0.5159 0.6759 0.0220 0.2119
uhle2004/ismir2004 0.4019 0.6868 0.0378 0.1721
tzanetakis2002/ismir2004_mmul 0.6371 0.7206 0.0583 0.2416
tzanetakis2002/ismir2004_msum 0.5418 0.7540 0.0568 0.2202
dixon2001/ismir2004_indu 0.4399 0.8120 0.0545 0.2042
tzanetakis2002/ismir2004_hist 0.8221 0.9669 0.1115 0.2571

Table 6: Mean OE1/OE2 for estimates compared to version 1.0 ordered by standard deviation.

CSV JSON LATEX PICKLE

Raw data OE1: CSV JSON LATEX PICKLE

Raw data OE2: CSV JSON LATEX PICKLE

OE1 distribution for 1.0

Figure 9: OE1 for estimates compared to version 1.0. Shown are the mean OE1 and an empirical distribution of the sample, using kernel density estimation (KDE).

CSV JSON LATEX PICKLE SVG PDF PNG

OE2 distribution for 1.0

Figure 10: OE2 for estimates compared to version 1.0. Shown are the mean OE2 and an empirical distribution of the sample, using kernel density estimation (KDE).

CSV JSON LATEX PICKLE SVG PDF PNG

Significance of Differences

Estimator alonso2004/ismir2004_corr alonso2004/ismir2004_spec boeck2015/tempodetector2016_default davies2009/mirex_qm_tempotracker dixon2001/ismir2004_indu dixon2001/ismir2004_trac dixon2003/ismir2004_auco echonest/version_3_2_1 gkiokas2012/default klapuri2006/ismir2004 klapuri2006/percival2014 oliveira2010/ibt percival2014/stem scheirer1998/ismir2004 scheirer1998/percival2014 schreiber2014/default schreiber2017/ismir2017 schreiber2017/mirex2017 schreiber2018/cnn schreiber2018/fcn schreiber2018/ismir2018 sun2021/default tzanetakis2002/ismir2004_hist tzanetakis2002/ismir2004_mmul tzanetakis2002/ismir2004_msum uhle2004/ismir2004 zplane/auftakt_v3
alonso2004/ismir2004_corr 1.0000 0.0000 0.0065 0.8245 0.0483 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0002 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0006 0.4914 0.0006 0.0000
alonso2004/ismir2004_spec 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0812 0.0000 0.0000 0.0000 0.0000 0.7187 0.4669 0.4902 0.0667 0.1749 0.0000 0.0005 0.2002 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0001
boeck2015/tempodetector2016_default 0.0065 0.0000 1.0000 0.0001 0.8091 0.0000 0.0000 0.0000 0.0002 0.0002 0.0001 0.2732 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0004 0.0000 0.0000 0.0000 0.0009 0.3070 0.0000
davies2009/mirex_qm_tempotracker 0.8245 0.0000 0.0001 1.0000 0.0070 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.5083 0.0000 0.0000
dixon2001/ismir2004_indu 0.0483 0.0000 0.8091 0.0070 1.0000 0.0000 0.0000 0.0000 0.0068 0.0055 0.0027 0.3245 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0001 0.0000 0.0106 0.0002 0.0000 0.0000 0.0039 0.2892 0.0001
dixon2001/ismir2004_trac 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0131 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.6127 0.0000 0.0000 0.0000 0.0000
dixon2003/ismir2004_auco 0.0000 0.0000 0.0000 0.0000 0.0000 0.0131 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.2741 0.0000 0.0000 0.0000 0.0000
echonest/version_3_2_1 0.0000 0.0812 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0068 0.0041 0.0093 0.9179 0.4202 0.0000 0.0000 0.0001 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
gkiokas2012/default 0.0000 0.0000 0.0002 0.0000 0.0068 0.0000 0.0000 0.0000 1.0000 0.7928 0.9671 0.0114 0.0000 0.0001 0.0000 0.0000 0.0000 0.0000 0.0653 0.0000 0.8110 0.1174 0.0000 0.0000 0.0000 0.0576 0.1942
klapuri2006/ismir2004 0.0000 0.0000 0.0002 0.0000 0.0055 0.0000 0.0000 0.0000 0.7928 1.0000 0.1464 0.0001 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0193 0.0000 0.9892 0.0493 0.0000 0.0000 0.0000 0.0273 0.0282
klapuri2006/percival2014 0.0000 0.0000 0.0001 0.0000 0.0027 0.0000 0.0000 0.0000 0.9671 0.1464 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0428 0.0000 0.6976 0.0942 0.0000 0.0000 0.0000 0.0122 0.0791
oliveira2010/ibt 0.0002 0.0000 0.2732 0.0000 0.3245 0.0000 0.0000 0.0000 0.0114 0.0001 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0038 0.0000 0.0000 0.0000 0.0000 0.8272 0.0000
percival2014/stem 0.0000 0.7187 0.0000 0.0000 0.0000 0.0000 0.0000 0.0068 0.0000 0.0000 0.0000 0.0000 1.0000 0.3888 0.1155 0.0011 0.0141 0.0000 0.0000 0.2368 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
scheirer1998/ismir2004 0.0000 0.4669 0.0000 0.0000 0.0000 0.0000 0.0000 0.0041 0.0001 0.0000 0.0000 0.0000 0.3888 1.0000 0.4122 0.0018 0.0107 0.0000 0.0099 0.7712 0.0000 0.0062 0.0000 0.0000 0.0000 0.0000 0.0009
scheirer1998/percival2014 0.0000 0.4902 0.0000 0.0000 0.0000 0.0000 0.0000 0.0093 0.0000 0.0000 0.0000 0.0000 0.1155 0.4122 1.0000 0.0006 0.0053 0.0000 0.0052 0.6622 0.0000 0.0109 0.0000 0.0000 0.0000 0.0000 0.0002
schreiber2014/default 0.0000 0.0667 0.0000 0.0000 0.0000 0.0000 0.0000 0.9179 0.0000 0.0000 0.0000 0.0000 0.0011 0.0018 0.0006 1.0000 0.3250 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2017/ismir2017 0.0000 0.1749 0.0000 0.0000 0.0000 0.0000 0.0000 0.4202 0.0000 0.0000 0.0000 0.0000 0.0141 0.0107 0.0053 0.3250 1.0000 0.0000 0.0000 0.0009 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2017/mirex2017 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2018/cnn 0.0000 0.0005 0.0000 0.0000 0.0001 0.0000 0.0000 0.0000 0.0653 0.0193 0.0428 0.0000 0.0000 0.0099 0.0052 0.0000 0.0000 0.0000 1.0000 0.0019 0.0071 0.8150 0.0000 0.0000 0.0000 0.0004 0.5239
schreiber2018/fcn 0.0000 0.2002 0.0000 0.0000 0.0000 0.0000 0.0000 0.0001 0.0000 0.0000 0.0000 0.0000 0.2368 0.7712 0.6622 0.0000 0.0009 0.0000 0.0019 1.0000 0.0000 0.0029 0.0000 0.0000 0.0000 0.0000 0.0014
schreiber2018/ismir2018 0.0000 0.0000 0.0004 0.0000 0.0106 0.0000 0.0000 0.0000 0.8110 0.9892 0.6976 0.0038 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0071 0.0000 1.0000 0.0330 0.0000 0.0000 0.0000 0.0542 0.0660
sun2021/default 0.0000 0.0000 0.0000 0.0000 0.0002 0.0000 0.0000 0.0000 0.1174 0.0493 0.0942 0.0000 0.0000 0.0062 0.0109 0.0000 0.0000 0.0000 0.8150 0.0029 0.0330 1.0000 0.0000 0.0000 0.0000 0.0014 0.6944
tzanetakis2002/ismir2004_hist 0.0000 0.0000 0.0000 0.0000 0.0000 0.6127 0.2741 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000
tzanetakis2002/ismir2004_mmul 0.0006 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000
tzanetakis2002/ismir2004_msum 0.4914 0.0000 0.0009 0.5083 0.0039 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000
uhle2004/ismir2004 0.0006 0.0000 0.3070 0.0000 0.2892 0.0000 0.0000 0.0000 0.0576 0.0273 0.0122 0.8272 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0004 0.0000 0.0542 0.0014 0.0000 0.0000 0.0000 1.0000 0.0009
zplane/auftakt_v3 0.0000 0.0001 0.0000 0.0000 0.0001 0.0000 0.0000 0.0000 0.1942 0.0282 0.0791 0.0000 0.0000 0.0009 0.0002 0.0000 0.0000 0.0000 0.5239 0.0014 0.0660 0.6944 0.0000 0.0000 0.0000 0.0009 1.0000

Table 7: Paired t-test p-values, using reference annotations 1.0 as groundtruth with OE1. H0: the true mean difference between paired samples is zero. If p<=ɑ, reject H0, i.e. we have a significant difference between estimates from the two algorithms. In the table, p-values<0.05 are set in bold.

CSV JSON LATEX PICKLE

Estimator alonso2004/ismir2004_corr alonso2004/ismir2004_spec boeck2015/tempodetector2016_default davies2009/mirex_qm_tempotracker dixon2001/ismir2004_indu dixon2001/ismir2004_trac dixon2003/ismir2004_auco echonest/version_3_2_1 gkiokas2012/default klapuri2006/ismir2004 klapuri2006/percival2014 oliveira2010/ibt percival2014/stem scheirer1998/ismir2004 scheirer1998/percival2014 schreiber2014/default schreiber2017/ismir2017 schreiber2017/mirex2017 schreiber2018/cnn schreiber2018/fcn schreiber2018/ismir2018 sun2021/default tzanetakis2002/ismir2004_hist tzanetakis2002/ismir2004_mmul tzanetakis2002/ismir2004_msum uhle2004/ismir2004 zplane/auftakt_v3
alonso2004/ismir2004_corr 1.0000 0.9298 0.0452 0.0274 0.0172 0.1051 0.1638 0.2900 0.1184 0.0089 0.0146 0.0809 0.0157 0.3171 0.2783 0.0012 0.0015 0.0121 0.0150 0.0110 0.0040 0.0013 0.0000 0.0153 0.0087 0.1636 0.0010
alonso2004/ismir2004_spec 0.9298 1.0000 0.0732 0.0049 0.0027 0.1560 0.0591 0.4128 0.2657 0.0202 0.0424 0.1301 0.0218 0.1381 0.4094 0.0014 0.0018 0.0241 0.0316 0.0202 0.0056 0.0032 0.0000 0.0032 0.0030 0.0899 0.0046
boeck2015/tempodetector2016_default 0.0452 0.0732 1.0000 0.0000 0.0000 0.0000 0.0000 0.2169 0.2892 0.2271 0.4482 0.9750 0.2674 0.0005 0.5037 0.0018 0.0027 0.2303 0.2831 0.2110 0.0181 0.0028 0.0000 0.0000 0.0000 0.0000 0.0141
davies2009/mirex_qm_tempotracker 0.0274 0.0049 0.0000 1.0000 0.4223 0.5739 0.3705 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.3026 0.0006 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.3323 0.3349 0.3654 0.0000
dixon2001/ismir2004_indu 0.0172 0.0027 0.0000 0.4223 1.0000 0.1334 0.1402 0.0001 0.0000 0.0000 0.0000 0.0000 0.0000 0.1542 0.0005 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.7693 0.8546 0.1616 0.0000
dixon2001/ismir2004_trac 0.1051 0.1560 0.0000 0.5739 0.1334 1.0000 0.5862 0.0034 0.0001 0.0000 0.0000 0.0006 0.0000 0.8620 0.0158 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.2492 0.3385 0.7899 0.0000
dixon2003/ismir2004_auco 0.1638 0.0591 0.0000 0.3705 0.1402 0.5862 1.0000 0.0054 0.0002 0.0000 0.0000 0.0002 0.0000 0.7980 0.0117 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.1011 0.0931 0.9604 0.0000
echonest/version_3_2_1 0.2900 0.4128 0.2169 0.0000 0.0001 0.0034 0.0054 1.0000 0.5695 0.0267 0.0500 0.2996 0.0589 0.0200 0.7226 0.0014 0.0016 0.0484 0.0531 0.0384 0.0073 0.0016 0.0000 0.0004 0.0002 0.0071 0.0015
gkiokas2012/default 0.1184 0.2657 0.2892 0.0000 0.0000 0.0001 0.0002 0.5695 1.0000 0.0275 0.0982 0.5626 0.1024 0.0026 0.8134 0.0001 0.0001 0.0296 0.0413 0.0156 0.0031 0.0005 0.0000 0.0000 0.0000 0.0002 0.0015
klapuri2006/ismir2004 0.0089 0.0202 0.2271 0.0000 0.0000 0.0000 0.0000 0.0267 0.0275 1.0000 0.5640 0.3531 0.9705 0.0000 0.1212 0.1416 0.1537 0.6411 0.7870 0.9152 0.4563 0.2339 0.0000 0.0000 0.0000 0.0000 0.1222
klapuri2006/percival2014 0.0146 0.0424 0.4482 0.0000 0.0000 0.0000 0.0000 0.0500 0.0982 0.5640 1.0000 0.5435 0.8068 0.0001 0.2471 0.0563 0.0633 0.9719 0.8718 0.7299 0.2338 0.0700 0.0000 0.0000 0.0000 0.0000 0.0494
oliveira2010/ibt 0.0809 0.1301 0.9750 0.0000 0.0000 0.0006 0.0002 0.2996 0.5626 0.3531 0.5435 1.0000 0.4193 0.0021 0.3732 0.0438 0.0421 0.4974 0.4795 0.4294 0.1433 0.0581 0.0000 0.0000 0.0000 0.0003 0.0366
percival2014/stem 0.0157 0.0218 0.2674 0.0000 0.0000 0.0000 0.0000 0.0589 0.1024 0.9705 0.8068 0.4193 1.0000 0.0002 0.2577 0.1408 0.1409 0.7158 0.8312 0.9585 0.4366 0.2151 0.0000 0.0000 0.0000 0.0000 0.1547
scheirer1998/ismir2004 0.3171 0.1381 0.0005 0.3026 0.1542 0.8620 0.7980 0.0200 0.0026 0.0000 0.0001 0.0021 0.0002 1.0000 0.0179 0.0000 0.0000 0.0000 0.0001 0.0000 0.0000 0.0000 0.0000 0.1057 0.1042 0.8226 0.0000
scheirer1998/percival2014 0.2783 0.4094 0.5037 0.0006 0.0005 0.0158 0.0117 0.7226 0.8134 0.1212 0.2471 0.3732 0.2577 0.0179 1.0000 0.0198 0.0180 0.2292 0.2520 0.1389 0.0934 0.0276 0.0000 0.0002 0.0001 0.0115 0.0084
schreiber2014/default 0.0012 0.0014 0.0018 0.0000 0.0000 0.0000 0.0000 0.0014 0.0001 0.1416 0.0563 0.0438 0.1408 0.0000 0.0198 1.0000 0.9520 0.0139 0.0355 0.0820 0.4658 0.8768 0.0000 0.0000 0.0000 0.0000 0.7430
schreiber2017/ismir2017 0.0015 0.0018 0.0027 0.0000 0.0000 0.0000 0.0000 0.0016 0.0001 0.1537 0.0633 0.0421 0.1409 0.0000 0.0180 0.9520 1.0000 0.0152 0.0612 0.1029 0.5424 0.9230 0.0000 0.0000 0.0000 0.0000 0.6983
schreiber2017/mirex2017 0.0121 0.0241 0.2303 0.0000 0.0000 0.0000 0.0000 0.0484 0.0296 0.6411 0.9719 0.4974 0.7158 0.0000 0.2292 0.0139 0.0152 1.0000 0.8506 0.6895 0.1374 0.0236 0.0000 0.0000 0.0000 0.0000 0.0499
schreiber2018/cnn 0.0150 0.0316 0.2831 0.0000 0.0000 0.0000 0.0000 0.0531 0.0413 0.7870 0.8718 0.4795 0.8312 0.0001 0.2520 0.0355 0.0612 0.8506 1.0000 0.8140 0.1873 0.0539 0.0000 0.0000 0.0000 0.0000 0.1064
schreiber2018/fcn 0.0110 0.0202 0.2110 0.0000 0.0000 0.0000 0.0000 0.0384 0.0156 0.9152 0.7299 0.4294 0.9585 0.0000 0.1389 0.0820 0.1029 0.6895 0.8140 1.0000 0.2914 0.1381 0.0000 0.0000 0.0000 0.0000 0.1259
schreiber2018/ismir2018 0.0040 0.0056 0.0181 0.0000 0.0000 0.0000 0.0000 0.0073 0.0031 0.4563 0.2338 0.1433 0.4366 0.0000 0.0934 0.4658 0.5424 0.1374 0.1873 0.2914 1.0000 0.5792 0.0000 0.0000 0.0000 0.0000 0.4329
sun2021/default 0.0013 0.0032 0.0028 0.0000 0.0000 0.0000 0.0000 0.0016 0.0005 0.2339 0.0700 0.0581 0.2151 0.0000 0.0276 0.8768 0.9230 0.0236 0.0539 0.1381 0.5792 1.0000 0.0000 0.0000 0.0000 0.0000 0.6669
tzanetakis2002/ismir2004_hist 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0001 0.0000 0.0000 0.0000
tzanetakis2002/ismir2004_mmul 0.0153 0.0032 0.0000 0.3323 0.7693 0.2492 0.1011 0.0004 0.0000 0.0000 0.0000 0.0000 0.0000 0.1057 0.0002 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0001 1.0000 0.8931 0.1162 0.0000
tzanetakis2002/ismir2004_msum 0.0087 0.0030 0.0000 0.3349 0.8546 0.3385 0.0931 0.0002 0.0000 0.0000 0.0000 0.0000 0.0000 0.1042 0.0001 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.8931 1.0000 0.1011 0.0000
uhle2004/ismir2004 0.1636 0.0899 0.0000 0.3654 0.1616 0.7899 0.9604 0.0071 0.0002 0.0000 0.0000 0.0003 0.0000 0.8226 0.0115 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.1162 0.1011 1.0000 0.0000
zplane/auftakt_v3 0.0010 0.0046 0.0141 0.0000 0.0000 0.0000 0.0000 0.0015 0.0015 0.1222 0.0494 0.0366 0.1547 0.0000 0.0084 0.7430 0.6983 0.0499 0.1064 0.1259 0.4329 0.6669 0.0000 0.0000 0.0000 0.0000 1.0000

Table 8: Paired t-test p-values, using reference annotations 1.0 as groundtruth with OE2. H0: the true mean difference between paired samples is zero. If p<=ɑ, reject H0, i.e. we have a significant difference between estimates from the two algorithms. In the table, p-values<0.05 are set in bold.

CSV JSON LATEX PICKLE

OE1 on Tempo-Subsets

How well does an estimator perform, when only taking a subset of the reference annotations into account? The graphs show mean OE1 for reference subsets with tempi in [T-10,T+10] BPM. Note that the graphs do not show confidence intervals and that some values may be based on very few estimates.

OE1 on Tempo-Subsets for 1.0

Figure 11: Mean OE1 for estimates compared to version 1.0 for tempo intervals around T.

CSV JSON LATEX PICKLE SVG PDF PNG

OE2 on Tempo-Subsets

How well does an estimator perform, when only taking a subset of the reference annotations into account? The graphs show mean OE2 for reference subsets with tempi in [T-10,T+10] BPM. Note that the graphs do not show confidence intervals and that some values may be based on very few estimates.

OE2 on Tempo-Subsets for 1.0

Figure 12: Mean OE2 for estimates compared to version 1.0 for tempo intervals around T.

CSV JSON LATEX PICKLE SVG PDF PNG

Estimated OE1 for Tempo

When fitting a generalized additive model (GAM) to OE1-values and a ground truth, what OE1 can we expect with confidence?

Estimated OE1 for Tempo for 1.0

Predictions of GAMs trained on OE1 for estimates for reference 1.0.

Figure 13: OE1 predictions of a generalized additive model (GAM) fit to OE1 results for 1.0. The 95% confidence interval around the prediction is shaded in gray.

CSV JSON LATEX PICKLE SVG PDF PNG

Estimated OE2 for Tempo

When fitting a generalized additive model (GAM) to OE2-values and a ground truth, what OE2 can we expect with confidence?

Estimated OE2 for Tempo for 1.0

Predictions of GAMs trained on OE2 for estimates for reference 1.0.

Figure 14: OE2 predictions of a generalized additive model (GAM) fit to OE2 results for 1.0. The 95% confidence interval around the prediction is shaded in gray.

CSV JSON LATEX PICKLE SVG PDF PNG

AOE1 and AOE2

AOE1 is defined as absolute octave error between an estimate and a reference value: AOE1(E) = |log2(E/R)|.

AOE2 is the minimum of AOE1 allowing the octave errors 2, 3, 1/2, and 1/3: AOE2(E) = min(AOE1(E), AOE1(2E), AOE1(3E), AOE1(½E), AOE1(⅓E)).

Mean AOE1/AOE2 Results for 1.0

Estimator AOE1_MEAN AOE1_STDEV AOE2_MEAN AOE2_STDEV
schreiber2017/mirex2017 0.0886 0.2710 0.0184 0.0486
schreiber2014/default 0.2427 0.4155 0.0327 0.0895
schreiber2017/ismir2017 0.2984 0.4480 0.0322 0.0869
echonest/version_3_2_1 0.3142 0.4316 0.0676 0.1275
schreiber2018/fcn 0.3148 0.4829 0.0255 0.0730
percival2014/stem 0.3342 0.4536 0.0479 0.1093
schreiber2018/cnn 0.3447 0.4881 0.0263 0.0726
sun2021/default 0.3774 0.4750 0.0378 0.0853
zplane/auftakt_v3 0.3841 0.4781 0.0605 0.1268
klapuri2006/percival2014 0.3905 0.4920 0.0371 0.0986
klapuri2006/ismir2004 0.3978 0.4942 0.0365 0.1002
schreiber2018/ismir2018 0.3990 0.4960 0.0343 0.0925
scheirer1998/percival2014 0.4119 0.4659 0.1040 0.1616
gkiokas2012/default 0.4247 0.5132 0.0335 0.0883
oliveira2010/ibt 0.4511 0.4878 0.0746 0.1338
boeck2015/tempodetector2016_default 0.4599 0.5309 0.0248 0.0774
alonso2004/ismir2004_spec 0.4764 0.4707 0.0996 0.1504
scheirer1998/ismir2004 0.4896 0.4703 0.1099 0.1609
uhle2004/ismir2004 0.5597 0.5656 0.0931 0.1496
davies2009/mirex_qm_tempotracker 0.5616 0.5462 0.0722 0.1354
alonso2004/ismir2004_corr 0.6621 0.5333 0.1348 0.1650
tzanetakis2002/ismir2004_msum 0.7048 0.6044 0.1391 0.1800
dixon2001/ismir2004_indu 0.7130 0.5868 0.1196 0.1743
tzanetakis2002/ismir2004_mmul 0.7608 0.5884 0.1702 0.1811
dixon2001/ismir2004_trac 0.8342 0.5431 0.0907 0.1539
dixon2003/ismir2004_auco 0.8944 0.5221 0.0838 0.1517
tzanetakis2002/ismir2004_hist 1.0447 0.7208 0.1742 0.2195

Table 9: Mean AOE1/AOE2 for estimates compared to version 1.0 ordered by mean.

CSV JSON LATEX PICKLE

Raw data AOE1: CSV JSON LATEX PICKLE

Raw data AOE2: CSV JSON LATEX PICKLE

AOE1 distribution for 1.0

Figure 15: AOE1 for estimates compared to version 1.0. Shown are the mean AOE1 and an empirical distribution of the sample, using kernel density estimation (KDE).

CSV JSON LATEX PICKLE SVG PDF PNG

AOE2 distribution for 1.0

Figure 16: AOE2 for estimates compared to version 1.0. Shown are the mean AOE2 and an empirical distribution of the sample, using kernel density estimation (KDE).

CSV JSON LATEX PICKLE SVG PDF PNG

Significance of Differences

Estimator alonso2004/ismir2004_corr alonso2004/ismir2004_spec boeck2015/tempodetector2016_default davies2009/mirex_qm_tempotracker dixon2001/ismir2004_indu dixon2001/ismir2004_trac dixon2003/ismir2004_auco echonest/version_3_2_1 gkiokas2012/default klapuri2006/ismir2004 klapuri2006/percival2014 oliveira2010/ibt percival2014/stem scheirer1998/ismir2004 scheirer1998/percival2014 schreiber2014/default schreiber2017/ismir2017 schreiber2017/mirex2017 schreiber2018/cnn schreiber2018/fcn schreiber2018/ismir2018 sun2021/default tzanetakis2002/ismir2004_hist tzanetakis2002/ismir2004_mmul tzanetakis2002/ismir2004_msum uhle2004/ismir2004 zplane/auftakt_v3
alonso2004/ismir2004_corr 1.0000 0.0000 0.0000 0.0003 0.1068 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0016 0.1852 0.0004 0.0000
alonso2004/ismir2004_spec 0.0000 1.0000 0.4757 0.0093 0.0000 0.0000 0.0000 0.0000 0.0391 0.0035 0.0014 0.3446 0.0000 0.7640 0.0053 0.0000 0.0000 0.0000 0.0000 0.0000 0.0069 0.0002 0.0000 0.0000 0.0000 0.0114 0.0007
boeck2015/tempodetector2016_default 0.0000 0.4757 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.1132 0.0037 0.0010 0.6842 0.0000 0.2781 0.2223 0.0000 0.0000 0.0000 0.0000 0.0000 0.0058 0.0003 0.0000 0.0000 0.0000 0.0002 0.0006
davies2009/mirex_qm_tempotracker 0.0003 0.0093 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0060 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.9344 0.0000
dixon2001/ismir2004_indu 0.1068 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0881 0.7788 0.0000 0.0000
dixon2001/ismir2004_trac 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0316 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0004 0.0000 0.0000 0.0000
dixon2003/ismir2004_auco 0.0000 0.0000 0.0000 0.0000 0.0000 0.0316 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
echonest/version_3_2_1 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0010 0.0028 0.0000 0.3845 0.0000 0.0067 0.0006 0.4754 0.0000 0.2029 0.9811 0.0006 0.0081 0.0000 0.0000 0.0000 0.0000 0.0032
gkiokas2012/default 0.0000 0.0391 0.1132 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.2595 0.1416 0.2451 0.0001 0.0196 0.5817 0.0000 0.0000 0.0000 0.0008 0.0000 0.3025 0.0536 0.0000 0.0000 0.0000 0.0000 0.0742
klapuri2006/ismir2004 0.0000 0.0035 0.0037 0.0000 0.0000 0.0000 0.0000 0.0010 0.2595 1.0000 0.0988 0.0001 0.0002 0.0002 0.1933 0.0000 0.0000 0.0000 0.0132 0.0007 0.9467 0.3653 0.0000 0.0000 0.0000 0.0000 0.3822
klapuri2006/percival2014 0.0000 0.0014 0.0010 0.0000 0.0000 0.0000 0.0000 0.0028 0.1416 0.0988 1.0000 0.0000 0.0008 0.0000 0.0881 0.0000 0.0000 0.0000 0.0344 0.0021 0.5980 0.5826 0.0000 0.0000 0.0000 0.0000 0.7014
oliveira2010/ibt 0.0000 0.3446 0.6842 0.0000 0.0000 0.0000 0.0000 0.0000 0.2451 0.0001 0.0000 1.0000 0.0000 0.1193 0.1328 0.0000 0.0000 0.0000 0.0000 0.0000 0.0054 0.0008 0.0000 0.0000 0.0000 0.0000 0.0000
percival2014/stem 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.3845 0.0001 0.0002 0.0008 0.0000 1.0000 0.0000 0.0002 0.0000 0.0504 0.0000 0.6405 0.4433 0.0019 0.0481 0.0000 0.0000 0.0000 0.0000 0.0067
scheirer1998/ismir2004 0.0000 0.7640 0.2781 0.0060 0.0000 0.0000 0.0000 0.0000 0.0196 0.0002 0.0000 0.1193 0.0000 1.0000 0.0020 0.0000 0.0000 0.0000 0.0000 0.0000 0.0007 0.0000 0.0000 0.0000 0.0000 0.0125 0.0000
scheirer1998/percival2014 0.0000 0.0053 0.2223 0.0000 0.0000 0.0000 0.0000 0.0067 0.5817 0.1933 0.0881 0.1328 0.0002 0.0020 1.0000 0.0000 0.0000 0.0000 0.0093 0.0009 0.5538 0.1082 0.0000 0.0000 0.0000 0.0000 0.2415
schreiber2014/default 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0006 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0004 0.0000 0.0000 0.0009 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2017/ismir2017 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.4754 0.0000 0.0000 0.0000 0.0000 0.0504 0.0000 0.0000 0.0004 1.0000 0.0000 0.0345 0.4690 0.0000 0.0005 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2017/mirex2017 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2018/cnn 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.2029 0.0008 0.0132 0.0344 0.0000 0.6405 0.0000 0.0093 0.0000 0.0345 0.0000 1.0000 0.1479 0.0033 0.1199 0.0000 0.0000 0.0000 0.0000 0.0703
schreiber2018/fcn 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.9811 0.0000 0.0007 0.0021 0.0000 0.4433 0.0000 0.0009 0.0009 0.4690 0.0000 0.1479 1.0000 0.0003 0.0059 0.0000 0.0000 0.0000 0.0000 0.0042
schreiber2018/ismir2018 0.0000 0.0069 0.0058 0.0000 0.0000 0.0000 0.0000 0.0006 0.3025 0.9467 0.5980 0.0054 0.0019 0.0007 0.5538 0.0000 0.0000 0.0000 0.0033 0.0003 1.0000 0.2995 0.0000 0.0000 0.0000 0.0000 0.4255
sun2021/default 0.0000 0.0002 0.0003 0.0000 0.0000 0.0000 0.0000 0.0081 0.0536 0.3653 0.5826 0.0008 0.0481 0.0000 0.1082 0.0000 0.0005 0.0000 0.1199 0.0059 0.2995 1.0000 0.0000 0.0000 0.0000 0.0000 0.7873
tzanetakis2002/ismir2004_hist 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000
tzanetakis2002/ismir2004_mmul 0.0016 0.0000 0.0000 0.0000 0.0881 0.0004 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0034 0.0000 0.0000
tzanetakis2002/ismir2004_msum 0.1852 0.0000 0.0000 0.0000 0.7788 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0034 1.0000 0.0000 0.0000
uhle2004/ismir2004 0.0004 0.0114 0.0002 0.9344 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0125 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000
zplane/auftakt_v3 0.0000 0.0007 0.0006 0.0000 0.0000 0.0000 0.0000 0.0032 0.0742 0.3822 0.7014 0.0000 0.0067 0.0000 0.2415 0.0000 0.0000 0.0000 0.0703 0.0042 0.4255 0.7873 0.0000 0.0000 0.0000 0.0000 1.0000

Table 10: Paired t-test p-values, using reference annotations 1.0 as groundtruth with AOE1. H0: the true mean difference between paired samples is zero. If p<=ɑ, reject H0, i.e. we have a significant difference between estimates from the two algorithms. In the table, p-values<0.05 are set in bold.

CSV JSON LATEX PICKLE

Estimator alonso2004/ismir2004_corr alonso2004/ismir2004_spec boeck2015/tempodetector2016_default davies2009/mirex_qm_tempotracker dixon2001/ismir2004_indu dixon2001/ismir2004_trac dixon2003/ismir2004_auco echonest/version_3_2_1 gkiokas2012/default klapuri2006/ismir2004 klapuri2006/percival2014 oliveira2010/ibt percival2014/stem scheirer1998/ismir2004 scheirer1998/percival2014 schreiber2014/default schreiber2017/ismir2017 schreiber2017/mirex2017 schreiber2018/cnn schreiber2018/fcn schreiber2018/ismir2018 sun2021/default tzanetakis2002/ismir2004_hist tzanetakis2002/ismir2004_mmul tzanetakis2002/ismir2004_msum uhle2004/ismir2004 zplane/auftakt_v3
alonso2004/ismir2004_corr 1.0000 0.0001 0.0000 0.0000 0.1317 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0156 0.0016 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0004 0.0007 0.6557 0.0000 0.0000
alonso2004/ismir2004_spec 0.0001 1.0000 0.0000 0.0010 0.0582 0.3256 0.0352 0.0000 0.0000 0.0000 0.0000 0.0020 0.0000 0.3979 0.7233 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0002 0.4046 0.0000
boeck2015/tempodetector2016_default 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0398 0.0300 0.0184 0.0000 0.0000 0.0000 0.0000 0.0702 0.1007 0.0772 0.7217 0.8784 0.0268 0.0023 0.0000 0.0000 0.0000 0.0000 0.0000
davies2009/mirex_qm_tempotracker 0.0000 0.0010 0.0000 1.0000 0.0000 0.0042 0.1217 0.5181 0.0000 0.0000 0.0000 0.7276 0.0003 0.0000 0.0016 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0085 0.0980
dixon2001/ismir2004_indu 0.1317 0.0582 0.0000 0.0000 1.0000 0.0056 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.3403 0.0971 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0443 0.0044 0.0000
dixon2001/ismir2004_trac 0.0000 0.3256 0.0000 0.0042 0.0056 1.0000 0.0871 0.0007 0.0000 0.0000 0.0000 0.0185 0.0000 0.0760 0.1475 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.8015 0.0001
dixon2003/ismir2004_auco 0.0000 0.0352 0.0000 0.1217 0.0000 0.0871 1.0000 0.0368 0.0000 0.0000 0.0000 0.2495 0.0000 0.0062 0.0890 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.2619 0.0021
echonest/version_3_2_1 0.0000 0.0000 0.0000 0.5181 0.0000 0.0007 0.0368 1.0000 0.0000 0.0000 0.0000 0.2722 0.0022 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0013 0.2678
gkiokas2012/default 0.0000 0.0000 0.0398 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.5318 0.4384 0.0000 0.0102 0.0000 0.0000 0.8845 0.8023 0.0002 0.0707 0.0416 0.8486 0.3322 0.0000 0.0000 0.0000 0.0000 0.0000
klapuri2006/ismir2004 0.0000 0.0000 0.0300 0.0000 0.0000 0.0000 0.0000 0.0000 0.5318 1.0000 0.8027 0.0000 0.0452 0.0000 0.0000 0.4626 0.4034 0.0000 0.0238 0.0196 0.6729 0.7840 0.0000 0.0000 0.0000 0.0000 0.0000
klapuri2006/percival2014 0.0000 0.0000 0.0184 0.0000 0.0000 0.0000 0.0000 0.0000 0.4384 0.8027 1.0000 0.0000 0.0555 0.0000 0.0000 0.3714 0.3235 0.0000 0.0157 0.0091 0.5708 0.8792 0.0000 0.0000 0.0000 0.0000 0.0000
oliveira2010/ibt 0.0000 0.0020 0.0000 0.7276 0.0000 0.0185 0.2495 0.2722 0.0000 0.0000 0.0000 1.0000 0.0001 0.0001 0.0009 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0203 0.0203
percival2014/stem 0.0000 0.0000 0.0000 0.0003 0.0000 0.0000 0.0000 0.0022 0.0102 0.0452 0.0555 0.0001 1.0000 0.0000 0.0000 0.0032 0.0020 0.0000 0.0001 0.0000 0.0106 0.0595 0.0000 0.0000 0.0000 0.0000 0.0355
scheirer1998/ismir2004 0.0156 0.3979 0.0000 0.0000 0.3403 0.0760 0.0062 0.0000 0.0000 0.0000 0.0000 0.0001 0.0000 1.0000 0.2380 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0060 0.0736 0.0000
scheirer1998/percival2014 0.0016 0.7233 0.0000 0.0016 0.0971 0.1475 0.0890 0.0000 0.0000 0.0000 0.0000 0.0009 0.0000 0.2380 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0005 0.2901 0.0000
schreiber2014/default 0.0000 0.0000 0.0702 0.0000 0.0000 0.0000 0.0000 0.0000 0.8845 0.4626 0.3714 0.0000 0.0032 0.0000 0.0000 1.0000 0.8608 0.0002 0.0997 0.0738 0.6960 0.1658 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2017/ismir2017 0.0000 0.0000 0.1007 0.0000 0.0000 0.0000 0.0000 0.0000 0.8023 0.4034 0.3235 0.0000 0.0020 0.0000 0.0000 0.8608 1.0000 0.0001 0.1844 0.1311 0.6528 0.1628 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2017/mirex2017 0.0000 0.0000 0.0772 0.0000 0.0000 0.0000 0.0000 0.0000 0.0002 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0002 0.0001 1.0000 0.0110 0.0415 0.0001 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2018/cnn 0.0000 0.0000 0.7217 0.0000 0.0000 0.0000 0.0000 0.0000 0.0707 0.0238 0.0157 0.0000 0.0001 0.0000 0.0000 0.0997 0.1844 0.0110 1.0000 0.8181 0.0330 0.0020 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2018/fcn 0.0000 0.0000 0.8784 0.0000 0.0000 0.0000 0.0000 0.0000 0.0416 0.0196 0.0091 0.0000 0.0000 0.0000 0.0000 0.0738 0.1311 0.0415 0.8181 1.0000 0.0192 0.0040 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2018/ismir2018 0.0000 0.0000 0.0268 0.0000 0.0000 0.0000 0.0000 0.0000 0.8486 0.6729 0.5708 0.0000 0.0106 0.0000 0.0000 0.6960 0.6528 0.0001 0.0330 0.0192 1.0000 0.4161 0.0000 0.0000 0.0000 0.0000 0.0000
sun2021/default 0.0000 0.0000 0.0023 0.0000 0.0000 0.0000 0.0000 0.0000 0.3322 0.7840 0.8792 0.0000 0.0595 0.0000 0.0000 0.1658 0.1628 0.0000 0.0020 0.0040 0.4161 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000
tzanetakis2002/ismir2004_hist 0.0004 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.6959 0.0000 0.0000 0.0000
tzanetakis2002/ismir2004_mmul 0.0007 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.6959 1.0000 0.0002 0.0000 0.0000
tzanetakis2002/ismir2004_msum 0.6557 0.0002 0.0000 0.0000 0.0443 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0060 0.0005 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0002 1.0000 0.0000 0.0000
uhle2004/ismir2004 0.0000 0.4046 0.0000 0.0085 0.0044 0.8015 0.2619 0.0013 0.0000 0.0000 0.0000 0.0203 0.0000 0.0736 0.2901 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0001
zplane/auftakt_v3 0.0000 0.0000 0.0000 0.0980 0.0000 0.0001 0.0021 0.2678 0.0000 0.0000 0.0000 0.0203 0.0355 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0001 1.0000

Table 11: Paired t-test p-values, using reference annotations 1.0 as groundtruth with AOE2. H0: the true mean difference between paired samples is zero. If p<=ɑ, reject H0, i.e. we have a significant difference between estimates from the two algorithms. In the table, p-values<0.05 are set in bold.

CSV JSON LATEX PICKLE

AOE1 on Tempo-Subsets

How well does an estimator perform, when only taking a subset of the reference annotations into account? The graphs show mean AOE1 for reference subsets with tempi in [T-10,T+10] BPM. Note that the graphs do not show confidence intervals and that some values may be based on very few estimates.

AOE1 on Tempo-Subsets for 1.0

Figure 17: Mean AOE1 for estimates compared to version 1.0 for tempo intervals around T.

CSV JSON LATEX PICKLE SVG PDF PNG

AOE2 on Tempo-Subsets

How well does an estimator perform, when only taking a subset of the reference annotations into account? The graphs show mean AOE2 for reference subsets with tempi in [T-10,T+10] BPM. Note that the graphs do not show confidence intervals and that some values may be based on very few estimates.

AOE2 on Tempo-Subsets for 1.0

Figure 18: Mean AOE2 for estimates compared to version 1.0 for tempo intervals around T.

CSV JSON LATEX PICKLE SVG PDF PNG

Estimated AOE1 for Tempo

When fitting a generalized additive model (GAM) to AOE1-values and a ground truth, what AOE1 can we expect with confidence?

Estimated AOE1 for Tempo for 1.0

Predictions of GAMs trained on AOE1 for estimates for reference 1.0.

Figure 19: AOE1 predictions of a generalized additive model (GAM) fit to AOE1 results for 1.0. The 95% confidence interval around the prediction is shaded in gray.

CSV JSON LATEX PICKLE SVG PDF PNG

Estimated AOE2 for Tempo

When fitting a generalized additive model (GAM) to AOE2-values and a ground truth, what AOE2 can we expect with confidence?

Estimated AOE2 for Tempo for 1.0

Predictions of GAMs trained on AOE2 for estimates for reference 1.0.

Figure 20: AOE2 predictions of a generalized additive model (GAM) fit to AOE2 results for 1.0. The 95% confidence interval around the prediction is shaded in gray.

CSV JSON LATEX PICKLE SVG PDF PNG


Generated by tempo_eval 0.1.1 on 2022-06-29 18:48. Size L.