gtzan

This is the tempo_eval report for the ‘gtzan’ corpus.

Reports for other corpora may be found here.

Table of Contents

References for ‘gtzan’

References

1.0

Attribute Value
Corpus GTZAN
Version 1.0
Curator George Tzanetakis
Annotator, bibtex Tzanetakis2013
Annotator, ref_url http://www.marsyas.info/tempo/

2.0

Attribute Value
Corpus GTZAN
Version 2.0
Curator Graham Percival
Annotator, bibtex Percival2014
Annotator, ref_url http://www.marsyas.info/tempo/

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI

Attribute Value
Corpus GTZAN
Version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI
Curator Ugo Marchand & Quentin Fresnel
Data Source manual annotation
Annotation Tools derived from beat annotations
Annotation Rules median of inter beat intervals (IBI)
Annotator, bibtex Marchand2015
Annotator, ref_url https://hal.archives-ouvertes.fr/hal-01252603/document

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI

Attribute Value
Corpus GTZAN
Version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI
Curator Ugo Marchand & Quentin Fresnel
Data Source manual annotation
Annotation Tools derived from beat annotations
Annotation Rules median of inter corresponding beat intervals (ICBI)
Annotator, bibtex Marchand2015
Annotator, ref_url https://hal.archives-ouvertes.fr/hal-01252603/document

Basic Statistics

Reference Size Min Max Avg Stdev Sweet Oct. Start Sweet Oct. Coverage
1.0 1000 38.00 168.00 94.27 24.45 66.00 0.81
2.0 999 38.00 168.00 94.92 24.41 66.00 0.81
GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI 1000 37.73 338.88 119.57 40.16 80.00 0.74
GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI 1000 37.68 339.29 119.53 40.13 79.00 0.74

Table 1: Basic statistics.

CSV JSON LATEX PICKLE

Smoothed Tempo Distribution

Figure 1: Percentage of values in tempo interval.

CSV JSON LATEX PICKLE SVG PDF PNG

Tag Distribution for ‘tag_gtzan’

Figure 2: Percentage of tracks tagged with tags from namespace ‘tag_gtzan’. Annotations are from reference 1.0.

CSV JSON LATEX PICKLE SVG PDF PNG

Tag Distribution for ‘tag_open’

Figure 3: Percentage of tracks tagged with tags from namespace ‘tag_open’. Annotations are from reference GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28.

CSV JSON LATEX PICKLE SVG PDF PNG

Beat-Based Tempo Variation

Figure 4: Fraction of the dataset with beat-annotated tracks with cvar < τ.

CSV JSON LATEX PICKLE SVG PDF PNG

Estimates for ‘gtzan’

Estimators

boeck2015/tempodetector2016_default

Attribute Value
Corpus gtzan
Version 0.17.dev0
Annotation Tools TempoDetector.2016, madmom, https://github.com/CPJKU/madmom
Annotator, bibtex Boeck2015

boeck2019/multi_task

Attribute Value
Corpus gtzan
Version 0.0.1
Annotation Tools model=multi_task, https://github.com/superbock/ISMIR2019
Annotator, bibtex Boeck2019

boeck2019/multi_task_hjdb

Attribute Value
Corpus gtzan
Version 0.0.1
Annotation Tools model=multi_task_hjdb, https://github.com/superbock/ISMIR2019
Annotator, bibtex Boeck2019

boeck2020/dar

Attribute Value
Corpus gtzan
Version 0.0.1
Annotation Tools https://github.com/superbock/ISMIR2020
Annotator, bibtex Boeck2020

davies2009/mirex_qm_tempotracker

Attribute Value  
Corpus gtzan  
Version 1.0  
Annotation Tools QM Tempotracker, Sonic Annotator plugin. https://code.soundsoftware.ac.uk/projects/mirex2013/repository/show/audio_tempo_estimation/qm-tempotracker Note that the current macOS build of ‘qm-vamp-plugins’ was used.  
Annotator, bibtex Davies2009 Davies2007

echonest/version_3_2_1

Attribute Value
Corpus gtzan
Version 3.2.1
Data Source Graham Percival and George Tzanetakis. Streamlined tempo estimation based on autocorrelation and crosscorrelation with pulses. IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP), 22(12):1765–1776, 2014.
Annotation Tools Echo Nest track analyzer v3.2.1
Annotator, bibtex Percival2014

gkiokas2012/default

Attribute Value
Corpus gtzan
Version 1.0
Data Source Graham Percival and George Tzanetakis. Streamlined tempo estimation based on autocorrelation and crosscorrelation with pulses. IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP), 22(12):1765–1776, 2014.
Annotation Tools Gkiokas2012
Annotator, bibtex Gkiokas2012

klapuri2006/percival2014

Attribute Value
Corpus gtzan
Version 1.0
Data Source Graham Percival and George Tzanetakis. Streamlined tempo estimation based on autocorrelation and crosscorrelation with pulses. IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP), 22(12):1765–1776, 2014.
Annotation Tools Klapuri 2006
Annotator, bibtex Klapuri2006

oliveira2010/ibt

Attribute Value
Corpus gtzan
Version 1.0
Data Source Graham Percival and George Tzanetakis. Streamlined tempo estimation based on autocorrelation and crosscorrelation with pulses. IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP), 22(12):1765–1776, 2014.
Annotation Tools Oliveira 2010
Annotator, bibtex Oliveira2010

percival2014/stem

Attribute Value
Corpus gtzan
Version 1.0
Annotation Tools percival 2014, ‘tempo’ implementation from Marsyas, http://marsyas.info, git checkout tempo-stem
Annotator, bibtex Percival2014

scheirer1998/percival2014

Attribute Value
Corpus gtzan
Version 1.0
Data Source Graham Percival and George Tzanetakis. Streamlined tempo estimation based on autocorrelation and crosscorrelation with pulses. IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP), 22(12):1765–1776, 2014.
Annotation Tools Scheirer 1998
Annotator, bibtex Scheirer1998

schreiber2014/default

Attribute Value
Corpus gtzan
Version 0.0.1
Annotation Tools schreiber 2014, http://www.tagtraum.com/tempo_estimation.html
Annotator, bibtex Schreiber2014

schreiber2017/ismir2017

Attribute Value
Corpus gtzan
Version 0.0.4
Annotation Tools schreiber 2017, model=ismir2017, http://www.tagtraum.com/tempo_estimation.html
Annotator, bibtex Schreiber2017

schreiber2017/mirex2017

Attribute Value
Corpus gtzan
Version 0.0.4
Annotation Tools schreiber 2017, model=mirex2017, http://www.tagtraum.com/tempo_estimation.html
Annotator, bibtex Schreiber2017

schreiber2018/cnn

Attribute Value
Corpus gtzan
Version 0.0.2
Data Source Hendrik Schreiber, Meinard Müller. A Single-Step Approach to Musical Tempo Estimation Using a Convolutional Neural Network. In Proceedings of the 19th International Society for Music Information Retrieval Conference (ISMIR), Paris, France, Sept. 2018.
Annotation Tools schreiber tempo-cnn (model=cnn), https://github.com/hendriks73/tempo-cnn

schreiber2018/fcn

Attribute Value
Corpus gtzan
Version 0.0.2
Data Source Hendrik Schreiber, Meinard Müller. A Single-Step Approach to Musical Tempo Estimation Using a Convolutional Neural Network. In Proceedings of the 19th International Society for Music Information Retrieval Conference (ISMIR), Paris, France, Sept. 2018.
Annotation Tools schreiber tempo-cnn (model=fcn), https://github.com/hendriks73/tempo-cnn

schreiber2018/ismir2018

Attribute Value
Corpus gtzan
Version 0.0.2
Data Source Hendrik Schreiber, Meinard Müller. A Single-Step Approach to Musical Tempo Estimation Using a Convolutional Neural Network. In Proceedings of the 19th International Society for Music Information Retrieval Conference (ISMIR), Paris, France, Sept. 2018.
Annotation Tools schreiber tempo-cnn (model=ismir2018), https://github.com/hendriks73/tempo-cnn

sun2021/default

Attribute Value
Corpus gtzan
Version 0.0.2
Data Source Xiaoheng Sun, Qiqi He, Yongwei Gao, Wei Li. Musical Tempo Estimation Using a Multi-scale Network. in Proc. of the 22nd Int. Society for Music Information Retrieval Conf., Online, 2021
Annotation Tools https://github.com/Qqi-HE/TempoEstimation_MGANet
Annotator, bibtex Sun2021

zplane/auftakt_v3

Attribute Value
Corpus gtzan
Version 3.0
Data Source Graham Percival and George Tzanetakis. Streamlined tempo estimation based on autocorrelation and crosscorrelation with pulses. IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP), 22(12):1765–1776, 2014.
Annotation Tools zplane aufTAKT version 3.0, http://licensing.zplane.de/technology#auftakt
Annotator, bibtex Percival2014

Basic Statistics

Estimator Size Min Max Avg Stdev Sweet Oct. Start Sweet Oct. Coverage
boeck2015/tempodetector2016_default 1000 41.10 240.00 114.70 33.90 72.00 0.79
boeck2019/multi_task 999 35.12 205.35 99.69 27.40 67.00 0.81
boeck2019/multi_task_hjdb 999 35.24 205.22 98.14 27.45 67.00 0.80
boeck2020/dar 999 44.78 243.25 116.02 35.42 79.00 0.75
davies2009/mirex_qm_tempotracker 1000 63.02 258.40 122.69 27.27 84.00 0.90
echonest/version_3_2_1 999 50.00 199.68 104.28 27.52 67.00 0.80
gkiokas2012/default 1000 31.00 246.00 107.52 30.05 71.00 0.80
klapuri2006/percival2014 1000 62.64 161.50 110.85 20.30 76.00 0.95
oliveira2010/ibt 1000 80.00 161.00 116.43 20.75 81.00 1.00
percival2014/stem 1000 50.42 154.27 102.55 21.52 71.00 0.92
scheirer1998/percival2014 979 61.35 179.81 103.73 27.80 64.00 0.80
schreiber2014/default 1000 52.05 163.94 101.61 21.62 71.00 0.91
schreiber2017/ismir2017 1000 40.53 202.66 102.97 22.44 70.00 0.90
schreiber2017/mirex2017 1000 20.27 202.35 98.19 25.01 70.00 0.83
schreiber2018/cnn 1000 50.00 237.00 112.00 31.25 75.00 0.81
schreiber2018/fcn 1000 38.00 222.00 109.14 30.57 71.00 0.81
schreiber2018/ismir2018 1000 53.00 232.00 112.89 28.19 77.00 0.87
sun2021/default 999 41.00 240.00 114.60 33.27 79.00 0.79
zplane/auftakt_v3 1000 65.00 165.40 109.51 22.56 73.00 0.89

Table 2: Basic statistics.

CSV JSON LATEX PICKLE

Smoothed Tempo Distribution

Figure 5: Percentage of values in tempo interval.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy

Accuracy1 is defined as the percentage of correct estimates, allowing a 4% tolerance for individual BPM values.

Accuracy2 additionally permits estimates to be wrong by a factor of 2, 3, 1/2 or 1/3 (so-called octave errors).

See [Gouyon2006].

Note: When comparing accuracy values for different algorithms, keep in mind that an algorithm may have been trained on the test set or that the test set may have even been created using one of the tested algorithms.

Accuracy Results for 1.0

Estimator Accuracy1 Accuracy2
schreiber2017/mirex2017 0.8800 0.9440
boeck2019/multi_task_hjdb 0.7700 0.9360
percival2014/stem 0.7690 0.9280
schreiber2017/ismir2017 0.7660 0.9240
boeck2019/multi_task 0.7610 0.9340
schreiber2014/default 0.7600 0.9170
schreiber2018/fcn 0.7160 0.9270
gkiokas2012/default 0.7060 0.9200
schreiber2018/cnn 0.7020 0.9360
boeck2015/tempodetector2016_default 0.6930 0.9420
klapuri2006/percival2014 0.6900 0.9100
zplane/auftakt_v3 0.6790 0.8780
schreiber2018/ismir2018 0.6740 0.9200
echonest/version_3_2_1 0.6700 0.8570
boeck2020/dar 0.6680 0.9520
sun2021/default 0.6500 0.9130
oliveira2010/ibt 0.6010 0.8610
davies2009/mirex_qm_tempotracker 0.5870 0.8860
scheirer1998/percival2014 0.5580 0.7570

Table 3: Mean accuracy of estimates compared to version 1.0 with 4% tolerance ordered by Accuracy1.

CSV JSON LATEX PICKLE

Raw data Accuracy1: CSV JSON LATEX PICKLE

Raw data Accuracy2: CSV JSON LATEX PICKLE

Accuracy1 for 1.0

Figure 6: Mean Accuracy1 for estimates compared to version 1.0 depending on tolerance.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy2 for 1.0

Figure 7: Mean Accuracy2 for estimates compared to version 1.0 depending on tolerance.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy Results for 2.0

Estimator Accuracy1 Accuracy2
schreiber2017/mirex2017 0.8899 0.9600
percival2014/stem 0.7808 0.9419
schreiber2017/ismir2017 0.7808 0.9389
schreiber2014/default 0.7738 0.9339
boeck2019/multi_task_hjdb 0.7738 0.9499
boeck2019/multi_task 0.7688 0.9469
schreiber2018/fcn 0.7267 0.9379
schreiber2018/cnn 0.7167 0.9510
gkiokas2012/default 0.7167 0.9379
boeck2015/tempodetector2016_default 0.7067 0.9560
klapuri2006/percival2014 0.7037 0.9249
zplane/auftakt_v3 0.6887 0.8919
schreiber2018/ismir2018 0.6857 0.9329
echonest/version_3_2_1 0.6797 0.8699
boeck2020/dar 0.6777 0.9660
sun2021/default 0.6617 0.9249
oliveira2010/ibt 0.6096 0.8699
davies2009/mirex_qm_tempotracker 0.6006 0.9009
scheirer1998/percival2014 0.5666 0.7628

Table 4: Mean accuracy of estimates compared to version 2.0 with 4% tolerance ordered by Accuracy1.

CSV JSON LATEX PICKLE

Raw data Accuracy1: CSV JSON LATEX PICKLE

Raw data Accuracy2: CSV JSON LATEX PICKLE

Accuracy1 for 2.0

Figure 8: Mean Accuracy1 for estimates compared to version 2.0 depending on tolerance.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy2 for 2.0

Figure 9: Mean Accuracy2 for estimates compared to version 2.0 depending on tolerance.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy Results for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI

Estimator Accuracy1 Accuracy2
boeck2020/dar 0.8510 0.9620
sun2021/default 0.8040 0.9310
boeck2015/tempodetector2016_default 0.7810 0.9560
schreiber2018/ismir2018 0.7750 0.9350
schreiber2018/cnn 0.7700 0.9500
schreiber2018/fcn 0.7530 0.9520
davies2009/mirex_qm_tempotracker 0.7100 0.9220
schreiber2017/ismir2017 0.7050 0.9360
boeck2019/multi_task 0.7050 0.9530
klapuri2006/percival2014 0.7030 0.9280
oliveira2010/ibt 0.6910 0.8790
schreiber2014/default 0.6840 0.9360
percival2014/stem 0.6830 0.9450
zplane/auftakt_v3 0.6810 0.8920
boeck2019/multi_task_hjdb 0.6810 0.9520
schreiber2017/mirex2017 0.6770 0.9520
gkiokas2012/default 0.6630 0.9360
echonest/version_3_2_1 0.6590 0.8760
scheirer1998/percival2014 0.5180 0.7700

Table 5: Mean accuracy of estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI with 4% tolerance ordered by Accuracy1.

CSV JSON LATEX PICKLE

Raw data Accuracy1: CSV JSON LATEX PICKLE

Raw data Accuracy2: CSV JSON LATEX PICKLE

Accuracy1 for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI

Figure 10: Mean Accuracy1 for estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI depending on tolerance.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy2 for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI

Figure 11: Mean Accuracy2 for estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI depending on tolerance.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy Results for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI

Estimator Accuracy1 Accuracy2
boeck2020/dar 0.8520 0.9630
sun2021/default 0.8070 0.9350
boeck2015/tempodetector2016_default 0.7810 0.9580
schreiber2018/ismir2018 0.7740 0.9340
schreiber2018/cnn 0.7700 0.9490
schreiber2018/fcn 0.7540 0.9540
davies2009/mirex_qm_tempotracker 0.7080 0.9160
klapuri2006/percival2014 0.7050 0.9290
schreiber2017/ismir2017 0.7040 0.9350
boeck2019/multi_task 0.7040 0.9520
oliveira2010/ibt 0.6890 0.8750
schreiber2014/default 0.6850 0.9370
percival2014/stem 0.6820 0.9440
zplane/auftakt_v3 0.6800 0.8900
boeck2019/multi_task_hjdb 0.6790 0.9500
schreiber2017/mirex2017 0.6780 0.9510
gkiokas2012/default 0.6620 0.9340
echonest/version_3_2_1 0.6580 0.8730
scheirer1998/percival2014 0.5160 0.7690

Table 6: Mean accuracy of estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI with 4% tolerance ordered by Accuracy1.

CSV JSON LATEX PICKLE

Raw data Accuracy1: CSV JSON LATEX PICKLE

Raw data Accuracy2: CSV JSON LATEX PICKLE

Accuracy1 for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI

Figure 12: Mean Accuracy1 for estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI depending on tolerance.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy2 for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI

Figure 13: Mean Accuracy2 for estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI depending on tolerance.

CSV JSON LATEX PICKLE SVG PDF PNG

Differing Items

For which items did a given estimator not estimate a correct value with respect to a given ground truth? Are there items which are either very difficult, not suitable for the task, or incorrectly annotated and therefore never estimated correctly, regardless which estimator is used?

Differing Items Accuracy1

Items with different tempo annotations (Accuracy1, 4% tolerance) in different versions:

1.0 compared with boeck2015/tempodetector2016_default (307 differences): ‘blues.00008’ ‘blues.00011’ ‘blues.00016’ ‘blues.00023’ ‘blues.00025’ ‘blues.00031’ ‘blues.00032’ ‘blues.00033’ ‘blues.00035’ ‘blues.00036’ ‘blues.00037’ … CSV

1.0 compared with boeck2019/multi_task (239 differences): ‘blues.00008’ ‘blues.00017’ ‘blues.00023’ ‘blues.00029’ ‘blues.00030’ ‘blues.00031’ ‘blues.00033’ ‘blues.00037’ ‘blues.00038’ ‘blues.00042’ ‘blues.00047’ … CSV

1.0 compared with boeck2019/multi_task_hjdb (230 differences): ‘blues.00002’ ‘blues.00008’ ‘blues.00017’ ‘blues.00023’ ‘blues.00029’ ‘blues.00030’ ‘blues.00031’ ‘blues.00033’ ‘blues.00036’ ‘blues.00037’ ‘blues.00038’ … CSV

1.0 compared with boeck2020/dar (332 differences): ‘blues.00011’ ‘blues.00016’ ‘blues.00023’ ‘blues.00032’ ‘blues.00033’ ‘blues.00035’ ‘blues.00037’ ‘blues.00040’ ‘blues.00042’ ‘blues.00047’ ‘blues.00051’ … CSV

1.0 compared with davies2009/mirex_qm_tempotracker (413 differences): ‘blues.00006’ ‘blues.00007’ ‘blues.00008’ ‘blues.00009’ ‘blues.00011’ ‘blues.00023’ ‘blues.00025’ ‘blues.00029’ ‘blues.00030’ ‘blues.00032’ ‘blues.00034’ … CSV

1.0 compared with echonest/version_3_2_1 (330 differences): ‘blues.00002’ ‘blues.00005’ ‘blues.00011’ ‘blues.00017’ ‘blues.00020’ ‘blues.00022’ ‘blues.00023’ ‘blues.00029’ ‘blues.00030’ ‘blues.00031’ ‘blues.00032’ … CSV

1.0 compared with gkiokas2012/default (294 differences): ‘blues.00000’ ‘blues.00002’ ‘blues.00004’ ‘blues.00008’ ‘blues.00010’ ‘blues.00017’ ‘blues.00023’ ‘blues.00029’ ‘blues.00033’ ‘blues.00035’ ‘blues.00036’ … CSV

1.0 compared with klapuri2006/percival2014 (310 differences): ‘blues.00006’ ‘blues.00008’ ‘blues.00010’ ‘blues.00011’ ‘blues.00017’ ‘blues.00023’ ‘blues.00029’ ‘blues.00031’ ‘blues.00032’ ‘blues.00035’ ‘blues.00037’ … CSV

1.0 compared with oliveira2010/ibt (399 differences): ‘blues.00001’ ‘blues.00003’ ‘blues.00005’ ‘blues.00006’ ‘blues.00008’ ‘blues.00011’ ‘blues.00017’ ‘blues.00023’ ‘blues.00025’ ‘blues.00032’ ‘blues.00035’ … CSV

1.0 compared with percival2014/stem (231 differences): ‘blues.00002’ ‘blues.00003’ ‘blues.00008’ ‘blues.00010’ ‘blues.00017’ ‘blues.00023’ ‘blues.00029’ ‘blues.00030’ ‘blues.00031’ ‘blues.00032’ ‘blues.00035’ … CSV

1.0 compared with scheirer1998/percival2014 (442 differences): ‘blues.00000’ ‘blues.00004’ ‘blues.00006’ ‘blues.00008’ ‘blues.00010’ ‘blues.00011’ ‘blues.00013’ ‘blues.00014’ ‘blues.00017’ ‘blues.00019’ ‘blues.00020’ … CSV

1.0 compared with schreiber2014/default (240 differences): ‘blues.00001’ ‘blues.00002’ ‘blues.00003’ ‘blues.00005’ ‘blues.00008’ ‘blues.00010’ ‘blues.00011’ ‘blues.00017’ ‘blues.00023’ ‘blues.00029’ ‘blues.00031’ … CSV

1.0 compared with schreiber2017/ismir2017 (234 differences): ‘blues.00008’ ‘blues.00010’ ‘blues.00011’ ‘blues.00017’ ‘blues.00020’ ‘blues.00023’ ‘blues.00029’ ‘blues.00031’ ‘blues.00032’ ‘blues.00033’ ‘blues.00035’ … CSV

1.0 compared with schreiber2017/mirex2017 (120 differences): ‘blues.00017’ ‘blues.00021’ ‘blues.00023’ ‘blues.00029’ ‘blues.00031’ ‘blues.00035’ ‘blues.00042’ ‘blues.00047’ ‘blues.00069’ ‘blues.00076’ ‘blues.00077’ … CSV

1.0 compared with schreiber2018/cnn (298 differences): ‘blues.00006’ ‘blues.00011’ ‘blues.00015’ ‘blues.00017’ ‘blues.00022’ ‘blues.00023’ ‘blues.00029’ ‘blues.00032’ ‘blues.00033’ ‘blues.00035’ ‘blues.00037’ … CSV

1.0 compared with schreiber2018/fcn (284 differences): ‘blues.00001’ ‘blues.00010’ ‘blues.00011’ ‘blues.00017’ ‘blues.00023’ ‘blues.00029’ ‘blues.00032’ ‘blues.00033’ ‘blues.00035’ ‘blues.00037’ ‘blues.00040’ … CSV

1.0 compared with schreiber2018/ismir2018 (326 differences): ‘blues.00001’ ‘blues.00003’ ‘blues.00005’ ‘blues.00006’ ‘blues.00011’ ‘blues.00017’ ‘blues.00023’ ‘blues.00029’ ‘blues.00031’ ‘blues.00033’ ‘blues.00035’ … CSV

1.0 compared with sun2021/default (350 differences): ‘blues.00001’ ‘blues.00005’ ‘blues.00006’ ‘blues.00007’ ‘blues.00011’ ‘blues.00017’ ‘blues.00023’ ‘blues.00029’ ‘blues.00032’ ‘blues.00033’ ‘blues.00034’ … CSV

1.0 compared with zplane/auftakt_v3 (321 differences): ‘blues.00002’ ‘blues.00003’ ‘blues.00006’ ‘blues.00008’ ‘blues.00011’ ‘blues.00017’ ‘blues.00018’ ‘blues.00023’ ‘blues.00029’ ‘blues.00032’ ‘blues.00035’ … CSV

2.0 compared with boeck2015/tempodetector2016_default (293 differences): ‘blues.00008’ ‘blues.00011’ ‘blues.00016’ ‘blues.00025’ ‘blues.00031’ ‘blues.00032’ ‘blues.00033’ ‘blues.00035’ ‘blues.00036’ ‘blues.00037’ ‘blues.00040’ … CSV

2.0 compared with boeck2019/multi_task (231 differences): ‘blues.00008’ ‘blues.00017’ ‘blues.00029’ ‘blues.00030’ ‘blues.00031’ ‘blues.00033’ ‘blues.00037’ ‘blues.00038’ ‘blues.00042’ ‘blues.00047’ ‘blues.00052’ … CSV

2.0 compared with boeck2019/multi_task_hjdb (226 differences): ‘blues.00002’ ‘blues.00008’ ‘blues.00017’ ‘blues.00029’ ‘blues.00030’ ‘blues.00031’ ‘blues.00033’ ‘blues.00036’ ‘blues.00037’ ‘blues.00038’ ‘blues.00040’ … CSV

2.0 compared with boeck2020/dar (322 differences): ‘blues.00011’ ‘blues.00016’ ‘blues.00032’ ‘blues.00033’ ‘blues.00035’ ‘blues.00037’ ‘blues.00040’ ‘blues.00042’ ‘blues.00047’ ‘blues.00051’ ‘blues.00052’ … CSV

2.0 compared with davies2009/mirex_qm_tempotracker (399 differences): ‘blues.00006’ ‘blues.00007’ ‘blues.00008’ ‘blues.00009’ ‘blues.00011’ ‘blues.00025’ ‘blues.00030’ ‘blues.00032’ ‘blues.00034’ ‘blues.00035’ ‘blues.00037’ … CSV

2.0 compared with echonest/version_3_2_1 (320 differences): ‘blues.00002’ ‘blues.00005’ ‘blues.00011’ ‘blues.00017’ ‘blues.00020’ ‘blues.00022’ ‘blues.00030’ ‘blues.00031’ ‘blues.00032’ ‘blues.00035’ ‘blues.00037’ … CSV

2.0 compared with gkiokas2012/default (283 differences): ‘blues.00000’ ‘blues.00002’ ‘blues.00004’ ‘blues.00008’ ‘blues.00010’ ‘blues.00017’ ‘blues.00033’ ‘blues.00035’ ‘blues.00036’ ‘blues.00037’ ‘blues.00040’ … CSV

2.0 compared with klapuri2006/percival2014 (296 differences): ‘blues.00006’ ‘blues.00008’ ‘blues.00010’ ‘blues.00011’ ‘blues.00017’ ‘blues.00031’ ‘blues.00032’ ‘blues.00035’ ‘blues.00037’ ‘blues.00042’ ‘blues.00047’ … CSV

2.0 compared with oliveira2010/ibt (390 differences): ‘blues.00001’ ‘blues.00003’ ‘blues.00005’ ‘blues.00006’ ‘blues.00008’ ‘blues.00011’ ‘blues.00017’ ‘blues.00025’ ‘blues.00032’ ‘blues.00035’ ‘blues.00037’ … CSV

2.0 compared with percival2014/stem (219 differences): ‘blues.00002’ ‘blues.00003’ ‘blues.00008’ ‘blues.00010’ ‘blues.00017’ ‘blues.00029’ ‘blues.00030’ ‘blues.00031’ ‘blues.00032’ ‘blues.00035’ ‘blues.00036’ … CSV

2.0 compared with scheirer1998/percival2014 (433 differences): ‘blues.00000’ ‘blues.00004’ ‘blues.00006’ ‘blues.00008’ ‘blues.00010’ ‘blues.00011’ ‘blues.00013’ ‘blues.00014’ ‘blues.00017’ ‘blues.00019’ ‘blues.00020’ … CSV

2.0 compared with schreiber2014/default (226 differences): ‘blues.00001’ ‘blues.00002’ ‘blues.00003’ ‘blues.00005’ ‘blues.00008’ ‘blues.00010’ ‘blues.00011’ ‘blues.00017’ ‘blues.00031’ ‘blues.00032’ ‘blues.00033’ … CSV

2.0 compared with schreiber2017/ismir2017 (219 differences): ‘blues.00008’ ‘blues.00010’ ‘blues.00011’ ‘blues.00017’ ‘blues.00020’ ‘blues.00031’ ‘blues.00032’ ‘blues.00033’ ‘blues.00035’ ‘blues.00037’ ‘blues.00040’ … CSV

2.0 compared with schreiber2017/mirex2017 (110 differences): ‘blues.00017’ ‘blues.00021’ ‘blues.00031’ ‘blues.00035’ ‘blues.00042’ ‘blues.00047’ ‘blues.00069’ ‘blues.00076’ ‘blues.00077’ ‘blues.00082’ ‘blues.00083’ … CSV

2.0 compared with schreiber2018/cnn (283 differences): ‘blues.00006’ ‘blues.00011’ ‘blues.00015’ ‘blues.00017’ ‘blues.00022’ ‘blues.00032’ ‘blues.00033’ ‘blues.00035’ ‘blues.00037’ ‘blues.00040’ ‘blues.00047’ … CSV

2.0 compared with schreiber2018/fcn (273 differences): ‘blues.00001’ ‘blues.00010’ ‘blues.00011’ ‘blues.00017’ ‘blues.00032’ ‘blues.00033’ ‘blues.00035’ ‘blues.00037’ ‘blues.00040’ ‘blues.00047’ ‘blues.00049’ … CSV

2.0 compared with schreiber2018/ismir2018 (314 differences): ‘blues.00001’ ‘blues.00003’ ‘blues.00005’ ‘blues.00006’ ‘blues.00011’ ‘blues.00017’ ‘blues.00031’ ‘blues.00033’ ‘blues.00035’ ‘blues.00037’ ‘blues.00042’ … CSV

2.0 compared with sun2021/default (338 differences): ‘blues.00001’ ‘blues.00005’ ‘blues.00006’ ‘blues.00007’ ‘blues.00011’ ‘blues.00017’ ‘blues.00032’ ‘blues.00033’ ‘blues.00034’ ‘blues.00035’ ‘blues.00037’ … CSV

2.0 compared with zplane/auftakt_v3 (311 differences): ‘blues.00002’ ‘blues.00003’ ‘blues.00006’ ‘blues.00008’ ‘blues.00011’ ‘blues.00017’ ‘blues.00018’ ‘blues.00032’ ‘blues.00035’ ‘blues.00037’ ‘blues.00038’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with boeck2015/tempodetector2016_default (219 differences): ‘blues.00008’ ‘blues.00010’ ‘blues.00011’ ‘blues.00016’ ‘blues.00025’ ‘blues.00031’ ‘blues.00032’ ‘blues.00033’ ‘blues.00036’ ‘blues.00038’ ‘blues.00042’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with boeck2019/multi_task (295 differences): ‘blues.00008’ ‘blues.00010’ ‘blues.00017’ ‘blues.00029’ ‘blues.00030’ ‘blues.00031’ ‘blues.00032’ ‘blues.00033’ ‘blues.00035’ ‘blues.00038’ ‘blues.00040’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with boeck2019/multi_task_hjdb (319 differences): ‘blues.00002’ ‘blues.00008’ ‘blues.00010’ ‘blues.00017’ ‘blues.00029’ ‘blues.00030’ ‘blues.00031’ ‘blues.00032’ ‘blues.00033’ ‘blues.00035’ ‘blues.00036’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with boeck2020/dar (149 differences): ‘blues.00010’ ‘blues.00011’ ‘blues.00016’ ‘blues.00032’ ‘blues.00033’ ‘blues.00038’ ‘blues.00042’ ‘blues.00044’ ‘blues.00045’ ‘blues.00052’ ‘blues.00056’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with davies2009/mirex_qm_tempotracker (290 differences): ‘blues.00008’ ‘blues.00009’ ‘blues.00010’ ‘blues.00011’ ‘blues.00025’ ‘blues.00030’ ‘blues.00034’ ‘blues.00038’ ‘blues.00040’ ‘blues.00042’ ‘blues.00044’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with echonest/version_3_2_1 (341 differences): ‘blues.00002’ ‘blues.00005’ ‘blues.00010’ ‘blues.00011’ ‘blues.00017’ ‘blues.00020’ ‘blues.00022’ ‘blues.00030’ ‘blues.00031’ ‘blues.00032’ ‘blues.00035’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with gkiokas2012/default (337 differences): ‘blues.00000’ ‘blues.00002’ ‘blues.00004’ ‘blues.00008’ ‘blues.00017’ ‘blues.00031’ ‘blues.00032’ ‘blues.00033’ ‘blues.00036’ ‘blues.00038’ ‘blues.00041’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with klapuri2006/percival2014 (297 differences): ‘blues.00006’ ‘blues.00008’ ‘blues.00011’ ‘blues.00017’ ‘blues.00031’ ‘blues.00038’ ‘blues.00040’ ‘blues.00042’ ‘blues.00044’ ‘blues.00045’ ‘blues.00049’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with oliveira2010/ibt (309 differences): ‘blues.00001’ ‘blues.00003’ ‘blues.00005’ ‘blues.00006’ ‘blues.00008’ ‘blues.00010’ ‘blues.00011’ ‘blues.00017’ ‘blues.00033’ ‘blues.00037’ ‘blues.00039’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with percival2014/stem (317 differences): ‘blues.00002’ ‘blues.00003’ ‘blues.00008’ ‘blues.00017’ ‘blues.00029’ ‘blues.00030’ ‘blues.00032’ ‘blues.00036’ ‘blues.00038’ ‘blues.00040’ ‘blues.00044’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with scheirer1998/percival2014 (482 differences): ‘blues.00000’ ‘blues.00004’ ‘blues.00008’ ‘blues.00011’ ‘blues.00012’ ‘blues.00013’ ‘blues.00017’ ‘blues.00019’ ‘blues.00020’ ‘blues.00023’ ‘blues.00029’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with schreiber2014/default (316 differences): ‘blues.00001’ ‘blues.00002’ ‘blues.00003’ ‘blues.00005’ ‘blues.00008’ ‘blues.00011’ ‘blues.00017’ ‘blues.00032’ ‘blues.00033’ ‘blues.00035’ ‘blues.00038’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with schreiber2017/ismir2017 (295 differences): ‘blues.00008’ ‘blues.00011’ ‘blues.00017’ ‘blues.00020’ ‘blues.00032’ ‘blues.00033’ ‘blues.00038’ ‘blues.00042’ ‘blues.00044’ ‘blues.00045’ ‘blues.00047’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with schreiber2017/mirex2017 (323 differences): ‘blues.00010’ ‘blues.00017’ ‘blues.00021’ ‘blues.00032’ ‘blues.00037’ ‘blues.00038’ ‘blues.00040’ ‘blues.00042’ ‘blues.00044’ ‘blues.00045’ ‘blues.00051’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with schreiber2018/cnn (230 differences): ‘blues.00006’ ‘blues.00010’ ‘blues.00011’ ‘blues.00015’ ‘blues.00017’ ‘blues.00022’ ‘blues.00032’ ‘blues.00033’ ‘blues.00038’ ‘blues.00044’ ‘blues.00045’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with schreiber2018/fcn (247 differences): ‘blues.00001’ ‘blues.00011’ ‘blues.00017’ ‘blues.00033’ ‘blues.00038’ ‘blues.00044’ ‘blues.00045’ ‘blues.00047’ ‘blues.00049’ ‘blues.00077’ ‘blues.00078’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with schreiber2018/ismir2018 (225 differences): ‘blues.00001’ ‘blues.00003’ ‘blues.00005’ ‘blues.00006’ ‘blues.00010’ ‘blues.00011’ ‘blues.00017’ ‘blues.00032’ ‘blues.00033’ ‘blues.00038’ ‘blues.00040’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with sun2021/default (196 differences): ‘blues.00001’ ‘blues.00005’ ‘blues.00006’ ‘blues.00010’ ‘blues.00011’ ‘blues.00017’ ‘blues.00032’ ‘blues.00033’ ‘blues.00034’ ‘blues.00037’ ‘blues.00038’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with zplane/auftakt_v3 (319 differences): ‘blues.00002’ ‘blues.00003’ ‘blues.00006’ ‘blues.00008’ ‘blues.00010’ ‘blues.00011’ ‘blues.00017’ ‘blues.00018’ ‘blues.00035’ ‘blues.00037’ ‘blues.00038’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with boeck2015/tempodetector2016_default (219 differences): ‘blues.00008’ ‘blues.00010’ ‘blues.00011’ ‘blues.00016’ ‘blues.00025’ ‘blues.00031’ ‘blues.00032’ ‘blues.00033’ ‘blues.00036’ ‘blues.00038’ ‘blues.00042’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with boeck2019/multi_task (296 differences): ‘blues.00008’ ‘blues.00010’ ‘blues.00017’ ‘blues.00029’ ‘blues.00030’ ‘blues.00031’ ‘blues.00032’ ‘blues.00033’ ‘blues.00035’ ‘blues.00038’ ‘blues.00040’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with boeck2019/multi_task_hjdb (321 differences): ‘blues.00002’ ‘blues.00008’ ‘blues.00010’ ‘blues.00017’ ‘blues.00029’ ‘blues.00030’ ‘blues.00031’ ‘blues.00032’ ‘blues.00033’ ‘blues.00035’ ‘blues.00036’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with boeck2020/dar (148 differences): ‘blues.00010’ ‘blues.00011’ ‘blues.00016’ ‘blues.00032’ ‘blues.00033’ ‘blues.00038’ ‘blues.00042’ ‘blues.00044’ ‘blues.00045’ ‘blues.00052’ ‘blues.00056’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with davies2009/mirex_qm_tempotracker (292 differences): ‘blues.00008’ ‘blues.00009’ ‘blues.00010’ ‘blues.00011’ ‘blues.00025’ ‘blues.00030’ ‘blues.00034’ ‘blues.00038’ ‘blues.00040’ ‘blues.00042’ ‘blues.00044’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with echonest/version_3_2_1 (342 differences): ‘blues.00002’ ‘blues.00005’ ‘blues.00010’ ‘blues.00011’ ‘blues.00017’ ‘blues.00020’ ‘blues.00022’ ‘blues.00030’ ‘blues.00031’ ‘blues.00032’ ‘blues.00035’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with gkiokas2012/default (338 differences): ‘blues.00000’ ‘blues.00002’ ‘blues.00004’ ‘blues.00008’ ‘blues.00017’ ‘blues.00031’ ‘blues.00032’ ‘blues.00033’ ‘blues.00036’ ‘blues.00038’ ‘blues.00041’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with klapuri2006/percival2014 (295 differences): ‘blues.00006’ ‘blues.00008’ ‘blues.00011’ ‘blues.00017’ ‘blues.00031’ ‘blues.00038’ ‘blues.00040’ ‘blues.00042’ ‘blues.00044’ ‘blues.00045’ ‘blues.00049’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with oliveira2010/ibt (311 differences): ‘blues.00001’ ‘blues.00003’ ‘blues.00005’ ‘blues.00006’ ‘blues.00008’ ‘blues.00010’ ‘blues.00011’ ‘blues.00017’ ‘blues.00037’ ‘blues.00039’ ‘blues.00040’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with percival2014/stem (318 differences): ‘blues.00002’ ‘blues.00003’ ‘blues.00008’ ‘blues.00017’ ‘blues.00029’ ‘blues.00030’ ‘blues.00032’ ‘blues.00036’ ‘blues.00038’ ‘blues.00040’ ‘blues.00044’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with scheirer1998/percival2014 (484 differences): ‘blues.00000’ ‘blues.00004’ ‘blues.00006’ ‘blues.00008’ ‘blues.00011’ ‘blues.00012’ ‘blues.00013’ ‘blues.00014’ ‘blues.00017’ ‘blues.00019’ ‘blues.00020’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with schreiber2014/default (315 differences): ‘blues.00001’ ‘blues.00002’ ‘blues.00003’ ‘blues.00005’ ‘blues.00008’ ‘blues.00011’ ‘blues.00017’ ‘blues.00032’ ‘blues.00033’ ‘blues.00035’ ‘blues.00038’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with schreiber2017/ismir2017 (296 differences): ‘blues.00008’ ‘blues.00011’ ‘blues.00017’ ‘blues.00020’ ‘blues.00032’ ‘blues.00033’ ‘blues.00038’ ‘blues.00042’ ‘blues.00044’ ‘blues.00045’ ‘blues.00047’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with schreiber2017/mirex2017 (322 differences): ‘blues.00010’ ‘blues.00017’ ‘blues.00021’ ‘blues.00032’ ‘blues.00037’ ‘blues.00038’ ‘blues.00040’ ‘blues.00042’ ‘blues.00044’ ‘blues.00045’ ‘blues.00051’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with schreiber2018/cnn (230 differences): ‘blues.00006’ ‘blues.00010’ ‘blues.00011’ ‘blues.00015’ ‘blues.00017’ ‘blues.00022’ ‘blues.00032’ ‘blues.00033’ ‘blues.00038’ ‘blues.00044’ ‘blues.00045’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with schreiber2018/fcn (246 differences): ‘blues.00001’ ‘blues.00011’ ‘blues.00017’ ‘blues.00033’ ‘blues.00038’ ‘blues.00044’ ‘blues.00045’ ‘blues.00047’ ‘blues.00049’ ‘blues.00077’ ‘blues.00078’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with schreiber2018/ismir2018 (226 differences): ‘blues.00001’ ‘blues.00003’ ‘blues.00005’ ‘blues.00006’ ‘blues.00010’ ‘blues.00011’ ‘blues.00017’ ‘blues.00032’ ‘blues.00033’ ‘blues.00038’ ‘blues.00040’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with sun2021/default (193 differences): ‘blues.00001’ ‘blues.00005’ ‘blues.00006’ ‘blues.00010’ ‘blues.00011’ ‘blues.00017’ ‘blues.00032’ ‘blues.00033’ ‘blues.00034’ ‘blues.00037’ ‘blues.00038’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with zplane/auftakt_v3 (320 differences): ‘blues.00002’ ‘blues.00003’ ‘blues.00006’ ‘blues.00008’ ‘blues.00010’ ‘blues.00011’ ‘blues.00017’ ‘blues.00018’ ‘blues.00035’ ‘blues.00037’ ‘blues.00038’ … CSV

None of the estimators estimated the following 6 items ‘correctly’ using Accuracy1: ‘classical.00036’ ‘classical.00067’ ‘classical.00080’ ‘jazz.00026’ ‘reggae.00098’ ‘reggae.00099’ CSV

Differing Items Accuracy2

Items with different tempo annotations (Accuracy2, 4% tolerance) in different versions:

1.0 compared with boeck2015/tempodetector2016_default (58 differences): ‘blues.00023’ ‘blues.00032’ ‘blues.00037’ ‘blues.00042’ ‘blues.00047’ ‘blues.00093’ ‘classical.00032’ ‘classical.00033’ ‘classical.00034’ ‘classical.00036’ ‘classical.00037’ … CSV

1.0 compared with boeck2019/multi_task (66 differences): ‘blues.00023’ ‘blues.00037’ ‘blues.00047’ ‘blues.00052’ ‘blues.00056’ ‘blues.00093’ ‘classical.00001’ ‘classical.00032’ ‘classical.00034’ ‘classical.00036’ ‘classical.00037’ … CSV

1.0 compared with boeck2019/multi_task_hjdb (64 differences): ‘blues.00023’ ‘blues.00037’ ‘blues.00047’ ‘blues.00052’ ‘blues.00056’ ‘blues.00093’ ‘classical.00001’ ‘classical.00032’ ‘classical.00034’ ‘classical.00036’ ‘classical.00037’ … CSV

1.0 compared with boeck2020/dar (48 differences): ‘blues.00023’ ‘blues.00032’ ‘blues.00037’ ‘blues.00052’ ‘classical.00032’ ‘classical.00033’ ‘classical.00034’ ‘classical.00036’ ‘classical.00037’ ‘classical.00039’ ‘classical.00041’ … CSV

1.0 compared with davies2009/mirex_qm_tempotracker (114 differences): ‘blues.00006’ ‘blues.00007’ ‘blues.00009’ ‘blues.00023’ ‘blues.00029’ ‘blues.00030’ ‘blues.00037’ ‘blues.00038’ ‘blues.00040’ ‘blues.00042’ ‘blues.00068’ … CSV

1.0 compared with echonest/version_3_2_1 (143 differences): ‘blues.00005’ ‘blues.00011’ ‘blues.00022’ ‘blues.00023’ ‘blues.00029’ ‘blues.00030’ ‘blues.00031’ ‘blues.00032’ ‘blues.00035’ ‘blues.00037’ ‘blues.00038’ … CSV

1.0 compared with gkiokas2012/default (80 differences): ‘blues.00023’ ‘blues.00029’ ‘blues.00036’ ‘blues.00037’ ‘blues.00042’ ‘blues.00047’ ‘blues.00093’ ‘classical.00003’ ‘classical.00009’ ‘classical.00019’ ‘classical.00027’ … CSV

1.0 compared with klapuri2006/percival2014 (90 differences): ‘blues.00006’ ‘blues.00011’ ‘blues.00023’ ‘blues.00029’ ‘blues.00031’ ‘blues.00032’ ‘blues.00037’ ‘blues.00042’ ‘blues.00056’ ‘blues.00069’ ‘blues.00072’ … CSV

1.0 compared with oliveira2010/ibt (139 differences): ‘blues.00001’ ‘blues.00003’ ‘blues.00005’ ‘blues.00006’ ‘blues.00011’ ‘blues.00023’ ‘blues.00025’ ‘blues.00035’ ‘blues.00037’ ‘blues.00038’ ‘blues.00039’ … CSV

1.0 compared with percival2014/stem (72 differences): ‘blues.00023’ ‘blues.00029’ ‘blues.00030’ ‘blues.00031’ ‘blues.00032’ ‘blues.00036’ ‘blues.00037’ ‘blues.00072’ ‘blues.00093’ ‘classical.00003’ ‘classical.00007’ … CSV

1.0 compared with scheirer1998/percival2014 (243 differences): ‘blues.00006’ ‘blues.00010’ ‘blues.00011’ ‘blues.00013’ ‘blues.00014’ ‘blues.00017’ ‘blues.00019’ ‘blues.00023’ ‘blues.00029’ ‘blues.00030’ ‘blues.00032’ … CSV

1.0 compared with schreiber2014/default (83 differences): ‘blues.00003’ ‘blues.00023’ ‘blues.00029’ ‘blues.00031’ ‘blues.00032’ ‘blues.00037’ ‘blues.00042’ ‘blues.00047’ ‘blues.00052’ ‘blues.00056’ ‘blues.00069’ … CSV

1.0 compared with schreiber2017/ismir2017 (76 differences): ‘blues.00023’ ‘blues.00029’ ‘blues.00031’ ‘blues.00032’ ‘blues.00037’ ‘blues.00047’ ‘blues.00052’ ‘blues.00056’ ‘blues.00069’ ‘blues.00093’ ‘blues.00096’ … CSV

1.0 compared with schreiber2017/mirex2017 (56 differences): ‘blues.00023’ ‘blues.00029’ ‘blues.00031’ ‘blues.00047’ ‘blues.00069’ ‘blues.00096’ ‘classical.00001’ ‘classical.00012’ ‘classical.00018’ ‘classical.00023’ ‘classical.00033’ … CSV

1.0 compared with schreiber2018/cnn (64 differences): ‘blues.00006’ ‘blues.00023’ ‘blues.00029’ ‘blues.00032’ ‘blues.00037’ ‘blues.00047’ ‘blues.00093’ ‘classical.00007’ ‘classical.00009’ ‘classical.00012’ ‘classical.00015’ … CSV

1.0 compared with schreiber2018/fcn (73 differences): ‘blues.00023’ ‘blues.00029’ ‘blues.00037’ ‘blues.00047’ ‘blues.00093’ ‘classical.00001’ ‘classical.00003’ ‘classical.00007’ ‘classical.00009’ ‘classical.00012’ ‘classical.00015’ … CSV

1.0 compared with schreiber2018/ismir2018 (80 differences): ‘blues.00006’ ‘blues.00011’ ‘blues.00023’ ‘blues.00029’ ‘blues.00031’ ‘blues.00037’ ‘blues.00042’ ‘blues.00047’ ‘blues.00056’ ‘blues.00093’ ‘classical.00007’ … CSV

1.0 compared with sun2021/default (87 differences): ‘blues.00005’ ‘blues.00007’ ‘blues.00023’ ‘blues.00029’ ‘blues.00032’ ‘blues.00037’ ‘blues.00042’ ‘blues.00047’ ‘blues.00052’ ‘blues.00056’ ‘blues.00089’ … CSV

1.0 compared with zplane/auftakt_v3 (122 differences): ‘blues.00011’ ‘blues.00023’ ‘blues.00029’ ‘blues.00032’ ‘blues.00035’ ‘blues.00037’ ‘blues.00038’ ‘blues.00042’ ‘blues.00052’ ‘blues.00056’ ‘blues.00063’ … CSV

2.0 compared with boeck2015/tempodetector2016_default (44 differences): ‘blues.00032’ ‘blues.00037’ ‘blues.00042’ ‘blues.00047’ ‘blues.00093’ ‘classical.00006’ ‘classical.00032’ ‘classical.00033’ ‘classical.00036’ ‘classical.00037’ ‘classical.00041’ … CSV

2.0 compared with boeck2019/multi_task (53 differences): ‘blues.00037’ ‘blues.00047’ ‘blues.00052’ ‘blues.00056’ ‘blues.00093’ ‘classical.00001’ ‘classical.00006’ ‘classical.00032’ ‘classical.00036’ ‘classical.00037’ ‘classical.00039’ … CSV

2.0 compared with boeck2019/multi_task_hjdb (50 differences): ‘blues.00037’ ‘blues.00047’ ‘blues.00052’ ‘blues.00056’ ‘blues.00093’ ‘classical.00001’ ‘classical.00006’ ‘classical.00032’ ‘classical.00036’ ‘classical.00037’ ‘classical.00039’ … CSV

2.0 compared with boeck2020/dar (34 differences): ‘blues.00032’ ‘blues.00037’ ‘blues.00052’ ‘classical.00006’ ‘classical.00032’ ‘classical.00033’ ‘classical.00036’ ‘classical.00037’ ‘classical.00039’ ‘classical.00041’ ‘classical.00043’ … CSV

2.0 compared with davies2009/mirex_qm_tempotracker (99 differences): ‘blues.00006’ ‘blues.00007’ ‘blues.00009’ ‘blues.00030’ ‘blues.00037’ ‘blues.00038’ ‘blues.00040’ ‘blues.00042’ ‘blues.00068’ ‘blues.00072’ ‘blues.00089’ … CSV

2.0 compared with echonest/version_3_2_1 (130 differences): ‘blues.00005’ ‘blues.00011’ ‘blues.00022’ ‘blues.00030’ ‘blues.00031’ ‘blues.00032’ ‘blues.00035’ ‘blues.00037’ ‘blues.00038’ ‘blues.00042’ ‘blues.00047’ … CSV

2.0 compared with gkiokas2012/default (62 differences): ‘blues.00036’ ‘blues.00037’ ‘blues.00042’ ‘blues.00047’ ‘blues.00093’ ‘classical.00003’ ‘classical.00009’ ‘classical.00031’ ‘classical.00032’ ‘classical.00036’ ‘classical.00037’ … CSV

2.0 compared with klapuri2006/percival2014 (75 differences): ‘blues.00006’ ‘blues.00011’ ‘blues.00031’ ‘blues.00032’ ‘blues.00037’ ‘blues.00042’ ‘blues.00056’ ‘blues.00069’ ‘blues.00072’ ‘blues.00093’ ‘classical.00003’ … CSV

2.0 compared with oliveira2010/ibt (130 differences): ‘blues.00001’ ‘blues.00003’ ‘blues.00005’ ‘blues.00006’ ‘blues.00011’ ‘blues.00025’ ‘blues.00035’ ‘blues.00037’ ‘blues.00038’ ‘blues.00039’ ‘blues.00042’ … CSV

2.0 compared with percival2014/stem (58 differences): ‘blues.00030’ ‘blues.00031’ ‘blues.00032’ ‘blues.00036’ ‘blues.00037’ ‘blues.00072’ ‘blues.00093’ ‘classical.00003’ ‘classical.00006’ ‘classical.00007’ ‘classical.00009’ … CSV

2.0 compared with scheirer1998/percival2014 (237 differences): ‘blues.00006’ ‘blues.00010’ ‘blues.00011’ ‘blues.00013’ ‘blues.00014’ ‘blues.00017’ ‘blues.00019’ ‘blues.00023’ ‘blues.00029’ ‘blues.00030’ ‘blues.00032’ … CSV

2.0 compared with schreiber2014/default (66 differences): ‘blues.00003’ ‘blues.00031’ ‘blues.00032’ ‘blues.00037’ ‘blues.00042’ ‘blues.00047’ ‘blues.00052’ ‘blues.00056’ ‘blues.00069’ ‘blues.00072’ ‘blues.00093’ … CSV

2.0 compared with schreiber2017/ismir2017 (61 differences): ‘blues.00031’ ‘blues.00032’ ‘blues.00037’ ‘blues.00047’ ‘blues.00052’ ‘blues.00056’ ‘blues.00069’ ‘blues.00093’ ‘blues.00096’ ‘classical.00001’ ‘classical.00006’ … CSV

2.0 compared with schreiber2017/mirex2017 (40 differences): ‘blues.00031’ ‘blues.00047’ ‘blues.00069’ ‘blues.00096’ ‘classical.00001’ ‘classical.00006’ ‘classical.00012’ ‘classical.00018’ ‘classical.00019’ ‘classical.00023’ ‘classical.00032’ … CSV

2.0 compared with schreiber2018/cnn (49 differences): ‘blues.00006’ ‘blues.00032’ ‘blues.00037’ ‘blues.00047’ ‘blues.00093’ ‘classical.00007’ ‘classical.00009’ ‘classical.00012’ ‘classical.00015’ ‘classical.00023’ ‘classical.00030’ … CSV

2.0 compared with schreiber2018/fcn (62 differences): ‘blues.00037’ ‘blues.00047’ ‘blues.00093’ ‘classical.00001’ ‘classical.00003’ ‘classical.00006’ ‘classical.00007’ ‘classical.00009’ ‘classical.00012’ ‘classical.00015’ ‘classical.00018’ … CSV

2.0 compared with schreiber2018/ismir2018 (67 differences): ‘blues.00006’ ‘blues.00011’ ‘blues.00031’ ‘blues.00037’ ‘blues.00042’ ‘blues.00047’ ‘blues.00056’ ‘blues.00093’ ‘classical.00006’ ‘classical.00007’ ‘classical.00009’ … CSV

2.0 compared with sun2021/default (75 differences): ‘blues.00005’ ‘blues.00007’ ‘blues.00032’ ‘blues.00037’ ‘blues.00042’ ‘blues.00047’ ‘blues.00052’ ‘blues.00056’ ‘blues.00089’ ‘blues.00093’ ‘classical.00006’ … CSV

2.0 compared with zplane/auftakt_v3 (108 differences): ‘blues.00011’ ‘blues.00032’ ‘blues.00035’ ‘blues.00037’ ‘blues.00038’ ‘blues.00042’ ‘blues.00052’ ‘blues.00056’ ‘blues.00063’ ‘blues.00069’ ‘blues.00072’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with boeck2015/tempodetector2016_default (44 differences): ‘blues.00032’ ‘blues.00038’ ‘blues.00042’ ‘blues.00092’ ‘blues.00093’ ‘classical.00009’ ‘classical.00032’ ‘classical.00033’ ‘classical.00034’ ‘classical.00036’ ‘classical.00037’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with boeck2019/multi_task (47 differences): ‘blues.00038’ ‘blues.00052’ ‘blues.00056’ ‘blues.00092’ ‘blues.00093’ ‘classical.00007’ ‘classical.00009’ ‘classical.00021’ ‘classical.00033’ ‘classical.00034’ ‘classical.00036’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with boeck2019/multi_task_hjdb (48 differences): ‘blues.00038’ ‘blues.00052’ ‘blues.00056’ ‘blues.00092’ ‘blues.00093’ ‘classical.00007’ ‘classical.00009’ ‘classical.00021’ ‘classical.00033’ ‘classical.00034’ ‘classical.00036’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with boeck2020/dar (38 differences): ‘blues.00032’ ‘blues.00038’ ‘blues.00052’ ‘blues.00092’ ‘blues.00093’ ‘classical.00007’ ‘classical.00009’ ‘classical.00033’ ‘classical.00034’ ‘classical.00036’ ‘classical.00037’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with davies2009/mirex_qm_tempotracker (78 differences): ‘blues.00009’ ‘blues.00030’ ‘blues.00040’ ‘blues.00042’ ‘blues.00072’ ‘blues.00089’ ‘blues.00092’ ‘blues.00093’ ‘blues.00099’ ‘classical.00001’ ‘classical.00003’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with echonest/version_3_2_1 (124 differences): ‘blues.00005’ ‘blues.00011’ ‘blues.00022’ ‘blues.00030’ ‘blues.00031’ ‘blues.00032’ ‘blues.00037’ ‘blues.00038’ ‘blues.00042’ ‘blues.00052’ ‘blues.00056’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with gkiokas2012/default (64 differences): ‘blues.00031’ ‘blues.00036’ ‘blues.00038’ ‘blues.00042’ ‘blues.00073’ ‘blues.00092’ ‘blues.00093’ ‘classical.00003’ ‘classical.00007’ ‘classical.00027’ ‘classical.00031’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with klapuri2006/percival2014 (72 differences): ‘blues.00006’ ‘blues.00011’ ‘blues.00031’ ‘blues.00038’ ‘blues.00042’ ‘blues.00056’ ‘blues.00069’ ‘blues.00072’ ‘blues.00092’ ‘classical.00003’ ‘classical.00006’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with oliveira2010/ibt (121 differences): ‘blues.00001’ ‘blues.00003’ ‘blues.00005’ ‘blues.00006’ ‘blues.00011’ ‘blues.00033’ ‘blues.00037’ ‘blues.00039’ ‘blues.00042’ ‘blues.00044’ ‘blues.00069’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with percival2014/stem (55 differences): ‘blues.00030’ ‘blues.00032’ ‘blues.00036’ ‘blues.00038’ ‘blues.00072’ ‘blues.00092’ ‘blues.00093’ ‘classical.00003’ ‘classical.00009’ ‘classical.00015’ ‘classical.00027’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with scheirer1998/percival2014 (230 differences): ‘blues.00011’ ‘blues.00012’ ‘blues.00013’ ‘blues.00019’ ‘blues.00023’ ‘blues.00029’ ‘blues.00030’ ‘blues.00032’ ‘blues.00034’ ‘blues.00035’ ‘blues.00037’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with schreiber2014/default (64 differences): ‘blues.00003’ ‘blues.00032’ ‘blues.00038’ ‘blues.00042’ ‘blues.00052’ ‘blues.00056’ ‘blues.00069’ ‘blues.00072’ ‘blues.00092’ ‘classical.00003’ ‘classical.00023’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with schreiber2017/ismir2017 (64 differences): ‘blues.00032’ ‘blues.00038’ ‘blues.00052’ ‘blues.00056’ ‘blues.00069’ ‘blues.00092’ ‘blues.00096’ ‘classical.00001’ ‘classical.00012’ ‘classical.00023’ ‘classical.00030’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with schreiber2017/mirex2017 (48 differences): ‘blues.00037’ ‘blues.00038’ ‘blues.00069’ ‘blues.00092’ ‘blues.00093’ ‘blues.00096’ ‘classical.00001’ ‘classical.00007’ ‘classical.00012’ ‘classical.00023’ ‘classical.00032’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with schreiber2018/cnn (50 differences): ‘blues.00032’ ‘blues.00038’ ‘blues.00092’ ‘classical.00001’ ‘classical.00009’ ‘classical.00015’ ‘classical.00021’ ‘classical.00023’ ‘classical.00030’ ‘classical.00032’ ‘classical.00033’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with schreiber2018/fcn (48 differences): ‘blues.00038’ ‘blues.00092’ ‘blues.00093’ ‘classical.00003’ ‘classical.00009’ ‘classical.00015’ ‘classical.00018’ ‘classical.00032’ ‘classical.00033’ ‘classical.00034’ ‘classical.00036’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with schreiber2018/ismir2018 (65 differences): ‘blues.00011’ ‘blues.00032’ ‘blues.00038’ ‘blues.00042’ ‘blues.00056’ ‘blues.00073’ ‘blues.00092’ ‘classical.00001’ ‘classical.00009’ ‘classical.00027’ ‘classical.00030’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with sun2021/default (69 differences): ‘blues.00005’ ‘blues.00032’ ‘blues.00037’ ‘blues.00038’ ‘blues.00042’ ‘blues.00052’ ‘blues.00056’ ‘blues.00089’ ‘blues.00092’ ‘blues.00093’ ‘classical.00007’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI compared with zplane/auftakt_v3 (108 differences): ‘blues.00011’ ‘blues.00035’ ‘blues.00038’ ‘blues.00042’ ‘blues.00052’ ‘blues.00056’ ‘blues.00063’ ‘blues.00069’ ‘blues.00072’ ‘blues.00092’ ‘blues.00093’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with boeck2015/tempodetector2016_default (42 differences): ‘blues.00032’ ‘blues.00038’ ‘blues.00042’ ‘blues.00092’ ‘blues.00093’ ‘classical.00009’ ‘classical.00032’ ‘classical.00033’ ‘classical.00034’ ‘classical.00036’ ‘classical.00037’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with boeck2019/multi_task (48 differences): ‘blues.00038’ ‘blues.00052’ ‘blues.00056’ ‘blues.00092’ ‘blues.00093’ ‘classical.00007’ ‘classical.00009’ ‘classical.00021’ ‘classical.00033’ ‘classical.00034’ ‘classical.00036’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with boeck2019/multi_task_hjdb (50 differences): ‘blues.00038’ ‘blues.00052’ ‘blues.00056’ ‘blues.00092’ ‘blues.00093’ ‘classical.00007’ ‘classical.00009’ ‘classical.00021’ ‘classical.00032’ ‘classical.00033’ ‘classical.00034’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with boeck2020/dar (37 differences): ‘blues.00032’ ‘blues.00038’ ‘blues.00052’ ‘blues.00092’ ‘blues.00093’ ‘classical.00007’ ‘classical.00033’ ‘classical.00034’ ‘classical.00036’ ‘classical.00037’ ‘classical.00039’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with davies2009/mirex_qm_tempotracker (84 differences): ‘blues.00009’ ‘blues.00030’ ‘blues.00040’ ‘blues.00042’ ‘blues.00072’ ‘blues.00089’ ‘blues.00092’ ‘blues.00093’ ‘blues.00099’ ‘classical.00001’ ‘classical.00003’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with echonest/version_3_2_1 (127 differences): ‘blues.00005’ ‘blues.00011’ ‘blues.00022’ ‘blues.00030’ ‘blues.00031’ ‘blues.00032’ ‘blues.00037’ ‘blues.00038’ ‘blues.00042’ ‘blues.00052’ ‘blues.00056’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with gkiokas2012/default (66 differences): ‘blues.00031’ ‘blues.00036’ ‘blues.00038’ ‘blues.00042’ ‘blues.00073’ ‘blues.00092’ ‘blues.00093’ ‘classical.00003’ ‘classical.00007’ ‘classical.00026’ ‘classical.00027’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with klapuri2006/percival2014 (71 differences): ‘blues.00006’ ‘blues.00011’ ‘blues.00031’ ‘blues.00038’ ‘blues.00042’ ‘blues.00056’ ‘blues.00069’ ‘blues.00072’ ‘blues.00092’ ‘classical.00003’ ‘classical.00006’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with oliveira2010/ibt (125 differences): ‘blues.00001’ ‘blues.00003’ ‘blues.00005’ ‘blues.00006’ ‘blues.00011’ ‘blues.00037’ ‘blues.00039’ ‘blues.00042’ ‘blues.00044’ ‘blues.00069’ ‘blues.00072’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with percival2014/stem (56 differences): ‘blues.00030’ ‘blues.00032’ ‘blues.00036’ ‘blues.00038’ ‘blues.00072’ ‘blues.00092’ ‘blues.00093’ ‘classical.00003’ ‘classical.00009’ ‘classical.00015’ ‘classical.00027’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with scheirer1998/percival2014 (231 differences): ‘blues.00006’ ‘blues.00011’ ‘blues.00012’ ‘blues.00013’ ‘blues.00014’ ‘blues.00019’ ‘blues.00023’ ‘blues.00029’ ‘blues.00030’ ‘blues.00032’ ‘blues.00034’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with schreiber2014/default (63 differences): ‘blues.00003’ ‘blues.00032’ ‘blues.00038’ ‘blues.00042’ ‘blues.00052’ ‘blues.00056’ ‘blues.00069’ ‘blues.00072’ ‘blues.00092’ ‘classical.00023’ ‘classical.00031’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with schreiber2017/ismir2017 (65 differences): ‘blues.00032’ ‘blues.00038’ ‘blues.00052’ ‘blues.00056’ ‘blues.00069’ ‘blues.00092’ ‘blues.00096’ ‘classical.00001’ ‘classical.00012’ ‘classical.00023’ ‘classical.00026’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with schreiber2017/mirex2017 (49 differences): ‘blues.00037’ ‘blues.00038’ ‘blues.00069’ ‘blues.00092’ ‘blues.00093’ ‘blues.00096’ ‘classical.00001’ ‘classical.00007’ ‘classical.00012’ ‘classical.00023’ ‘classical.00026’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with schreiber2018/cnn (51 differences): ‘blues.00006’ ‘blues.00032’ ‘blues.00038’ ‘blues.00092’ ‘classical.00001’ ‘classical.00009’ ‘classical.00015’ ‘classical.00021’ ‘classical.00023’ ‘classical.00030’ ‘classical.00032’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with schreiber2018/fcn (46 differences): ‘blues.00038’ ‘blues.00092’ ‘blues.00093’ ‘classical.00003’ ‘classical.00009’ ‘classical.00015’ ‘classical.00018’ ‘classical.00032’ ‘classical.00033’ ‘classical.00034’ ‘classical.00036’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with schreiber2018/ismir2018 (66 differences): ‘blues.00011’ ‘blues.00032’ ‘blues.00038’ ‘blues.00042’ ‘blues.00056’ ‘blues.00073’ ‘blues.00092’ ‘classical.00001’ ‘classical.00009’ ‘classical.00027’ ‘classical.00030’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with sun2021/default (65 differences): ‘blues.00005’ ‘blues.00032’ ‘blues.00037’ ‘blues.00038’ ‘blues.00042’ ‘blues.00052’ ‘blues.00056’ ‘blues.00089’ ‘blues.00092’ ‘blues.00093’ ‘classical.00007’ … CSV

GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI compared with zplane/auftakt_v3 (110 differences): ‘blues.00011’ ‘blues.00035’ ‘blues.00038’ ‘blues.00042’ ‘blues.00052’ ‘blues.00056’ ‘blues.00063’ ‘blues.00069’ ‘blues.00072’ ‘blues.00092’ ‘blues.00093’ … CSV

All tracks were estimated ‘correctly’ by at least one system.

Significance of Differences

Estimator boeck2015/tempodetector2016_default boeck2019/multi_task boeck2019/multi_task_hjdb boeck2020/dar davies2009/mirex_qm_tempotracker echonest/version_3_2_1 gkiokas2012/default klapuri2006/percival2014 oliveira2010/ibt percival2014/stem scheirer1998/percival2014 schreiber2014/default schreiber2017/ismir2017 schreiber2017/mirex2017 schreiber2018/cnn schreiber2018/fcn schreiber2018/ismir2018 sun2021/default zplane/auftakt_v3
boeck2015/tempodetector2016_default 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.4188 0.0546 0.6158 0.0412 0.0000
boeck2019/multi_task 0.0000 1.0000 0.0005 0.0000 0.8577 0.0020 0.0057 1.0000 0.3933 0.1134 0.0000 0.1268 0.9358 0.0595 0.0000 0.0005 0.0000 0.0000 0.1090
boeck2019/multi_task_hjdb 0.0000 0.0005 1.0000 0.0000 0.1077 0.1726 0.2861 0.1007 0.6009 0.8805 0.0000 0.6851 0.0633 1.0000 0.0000 0.0000 0.0000 0.0000 1.0000
boeck2020/dar 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0001 0.0000
davies2009/mirex_qm_tempotracker 0.0000 0.8577 0.1077 0.0000 1.0000 0.0014 0.0070 0.8681 0.1158 0.0948 0.0000 0.1649 0.8471 0.0908 0.0000 0.0033 0.0000 0.0000 0.0489
echonest/version_3_2_1 0.0000 0.0020 0.1726 0.0000 0.0014 1.0000 0.8352 0.0011 0.0455 0.1056 0.0000 0.0599 0.0014 0.1919 0.0000 0.0000 0.0000 0.0000 0.1454
gkiokas2012/default 0.0000 0.0057 0.2861 0.0000 0.0070 0.8352 1.0000 0.0045 0.1089 0.1636 0.0000 0.1058 0.0037 0.2960 0.0000 0.0000 0.0000 0.0000 0.2517
klapuri2006/percival2014 0.0000 1.0000 0.1007 0.0000 0.8681 0.0011 0.0045 1.0000 0.1745 0.0564 0.0000 0.1331 1.0000 0.0680 0.0000 0.0007 0.0000 0.0000 0.0360
oliveira2010/ibt 0.0000 0.3933 0.6009 0.0000 0.1158 0.0455 0.1089 0.1745 1.0000 0.6824 0.0000 0.8458 0.3506 0.5450 0.0000 0.0001 0.0000 0.0000 0.5150
percival2014/stem 0.0000 0.1134 0.8805 0.0000 0.0948 0.1056 0.1636 0.0564 0.6824 1.0000 0.0000 0.8613 0.0864 0.8090 0.0000 0.0000 0.0000 0.0000 0.9349
scheirer1998/percival2014 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2014/default 0.0000 0.1268 0.6851 0.0000 0.1649 0.0599 0.1058 0.1331 0.8458 0.8613 0.0000 1.0000 0.0761 0.6158 0.0000 0.0000 0.0000 0.0000 0.7610
schreiber2017/ismir2017 0.0000 0.9358 0.0633 0.0000 0.8471 0.0014 0.0037 1.0000 0.3506 0.0864 0.0000 0.0761 1.0000 0.0203 0.0000 0.0003 0.0000 0.0000 0.0725
schreiber2017/mirex2017 0.0000 0.0595 1.0000 0.0000 0.0908 0.1919 0.2960 0.0680 0.5450 0.8090 0.0000 0.6158 0.0203 1.0000 0.0000 0.0000 0.0000 0.0000 0.9436
schreiber2018/cnn 0.4188 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.2081 0.7984 0.0038 0.0000
schreiber2018/fcn 0.0546 0.0005 0.0000 0.0000 0.0033 0.0000 0.0000 0.0007 0.0001 0.0000 0.0000 0.0000 0.0003 0.0000 0.2081 1.0000 0.1306 0.0002 0.0000
schreiber2018/ismir2018 0.6158 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.7984 0.1306 1.0000 0.0066 0.0000
sun2021/default 0.0412 0.0000 0.0000 0.0001 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0038 0.0002 0.0066 1.0000 0.0000
zplane/auftakt_v3 0.0000 0.1090 1.0000 0.0000 0.0489 0.1454 0.2517 0.0360 0.5150 0.9349 0.0000 0.7610 0.0725 0.9436 0.0000 0.0000 0.0000 0.0000 1.0000

Table 7: McNemar p-values, using reference annotations GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI as groundtruth with Accuracy1 [Gouyon2006]. H0: both estimators disagree with the groundtruth to the same amount. If p<=ɑ, reject H0, i.e. we have a significant difference in the disagreement with the groundtruth. In the table, p-values<0.05 are set in bold.

CSV JSON LATEX PICKLE

Estimator boeck2015/tempodetector2016_default boeck2019/multi_task boeck2019/multi_task_hjdb boeck2020/dar davies2009/mirex_qm_tempotracker echonest/version_3_2_1 gkiokas2012/default klapuri2006/percival2014 oliveira2010/ibt percival2014/stem scheirer1998/percival2014 schreiber2014/default schreiber2017/ismir2017 schreiber2017/mirex2017 schreiber2018/cnn schreiber2018/fcn schreiber2018/ismir2018 sun2021/default zplane/auftakt_v3
boeck2015/tempodetector2016_default 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.4218 0.0454 0.6769 0.0715 0.0000
boeck2019/multi_task 0.0000 1.0000 0.0007 0.0000 0.8114 0.0019 0.0057 0.9452 0.4289 0.1134 0.0000 0.0898 0.9358 0.0418 0.0000 0.0009 0.0000 0.0000 0.1108
boeck2019/multi_task_hjdb 0.0000 0.0007 1.0000 0.0000 0.1089 0.1511 0.2581 0.1680 0.6009 0.9403 0.0000 0.8715 0.0760 0.8250 0.0000 0.0000 0.0000 0.0000 0.9465
boeck2020/dar 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0001 0.0000
davies2009/mirex_qm_tempotracker 0.0000 0.8114 0.1089 0.0000 1.0000 0.0012 0.0057 0.6158 0.1102 0.0803 0.0000 0.1138 0.7967 0.0607 0.0001 0.0061 0.0000 0.0000 0.0428
echonest/version_3_2_1 0.0000 0.0019 0.1511 0.0000 0.0012 1.0000 0.8344 0.0021 0.0409 0.1021 0.0000 0.0793 0.0012 0.2408 0.0000 0.0000 0.0000 0.0000 0.1454
gkiokas2012/default 0.0000 0.0057 0.2581 0.0000 0.0057 0.8344 1.0000 0.0083 0.0978 0.1636 0.0000 0.1414 0.0039 0.3674 0.0000 0.0000 0.0000 0.0000 0.2517
klapuri2006/percival2014 0.0000 0.9452 0.1680 0.0000 0.6158 0.0021 0.0083 1.0000 0.3193 0.1007 0.0000 0.1534 0.9358 0.0771 0.0000 0.0005 0.0000 0.0000 0.0655
oliveira2010/ibt 0.0000 0.4289 0.6009 0.0000 0.1102 0.0409 0.0978 0.3193 1.0000 0.6323 0.0000 0.6967 0.3851 0.4322 0.0000 0.0001 0.0000 0.0000 0.4654
percival2014/stem 0.0000 0.1134 0.9403 0.0000 0.0803 0.1021 0.1636 0.1007 0.6323 1.0000 0.0000 1.0000 0.0864 0.6870 0.0000 0.0000 0.0000 0.0000 0.9354
scheirer1998/percival2014 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2014/default 0.0000 0.0898 0.8715 0.0000 0.1138 0.0793 0.1414 0.1534 0.6967 1.0000 0.0000 1.0000 0.0466 0.6134 0.0000 0.0000 0.0000 0.0000 0.8791
schreiber2017/ismir2017 0.0000 0.9358 0.0760 0.0000 0.7967 0.0012 0.0039 0.9358 0.3851 0.0864 0.0000 0.0466 1.0000 0.0122 0.0000 0.0004 0.0000 0.0000 0.0742
schreiber2017/mirex2017 0.0000 0.0418 0.8250 0.0000 0.0607 0.2408 0.3674 0.0771 0.4322 0.6870 0.0000 0.6134 0.0122 1.0000 0.0000 0.0000 0.0000 0.0000 0.8320
schreiber2018/cnn 0.4218 0.0000 0.0000 0.0000 0.0001 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.1778 0.7325 0.0082 0.0000
schreiber2018/fcn 0.0454 0.0009 0.0000 0.0000 0.0061 0.0000 0.0000 0.0005 0.0001 0.0000 0.0000 0.0000 0.0004 0.0000 0.1778 1.0000 0.0948 0.0003 0.0000
schreiber2018/ismir2018 0.6769 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.7325 0.0948 1.0000 0.0176 0.0000
sun2021/default 0.0715 0.0000 0.0000 0.0001 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0082 0.0003 0.0176 1.0000 0.0000
zplane/auftakt_v3 0.0000 0.1108 0.9465 0.0000 0.0428 0.1454 0.2517 0.0655 0.4654 0.9354 0.0000 0.8791 0.0742 0.8320 0.0000 0.0000 0.0000 0.0000 1.0000

Table 8: McNemar p-values, using reference annotations GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI as groundtruth with Accuracy1 [Gouyon2006]. H0: both estimators disagree with the groundtruth to the same amount. If p<=ɑ, reject H0, i.e. we have a significant difference in the disagreement with the groundtruth. In the table, p-values<0.05 are set in bold.

CSV JSON LATEX PICKLE

Estimator boeck2015/tempodetector2016_default boeck2019/multi_task boeck2019/multi_task_hjdb boeck2020/dar davies2009/mirex_qm_tempotracker echonest/version_3_2_1 gkiokas2012/default klapuri2006/percival2014 oliveira2010/ibt percival2014/stem scheirer1998/percival2014 schreiber2014/default schreiber2017/ismir2017 schreiber2017/mirex2017 schreiber2018/cnn schreiber2018/fcn schreiber2018/ismir2018 sun2021/default zplane/auftakt_v3
boeck2015/tempodetector2016_default 1.0000 0.0000 0.0000 0.0176 0.0000 0.0885 0.5458 0.8855 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.4795 0.1613 0.0898 0.0003 0.2101
boeck2019/multi_task 0.0000 1.0000 0.5758 0.0000 0.0000 0.0000 0.0008 0.0000 0.0000 0.4249 0.0000 0.7431 0.3845 0.0000 0.0005 0.0045 0.0000 0.0000 0.0000
boeck2019/multi_task_hjdb 0.0000 0.5758 1.0000 0.0000 0.0000 0.0000 0.0003 0.0000 0.0000 0.6642 0.0000 0.9370 0.6464 0.0000 0.0003 0.0020 0.0000 0.0000 0.0000
boeck2020/dar 0.0176 0.0000 0.0000 1.0000 0.0000 0.9490 0.0178 0.0934 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0037 0.0005 0.5650 0.1883 0.5177
davies2009/mirex_qm_tempotracker 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.4595 0.0000 0.0518 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
echonest/version_3_2_1 0.0885 0.0000 0.0000 0.9490 0.0000 1.0000 0.0145 0.0935 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0184 0.0025 0.7372 0.2623 0.5836
gkiokas2012/default 0.5458 0.0008 0.0003 0.0178 0.0000 0.0145 1.0000 0.4109 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.9470 0.5422 0.0465 0.0005 0.0700
klapuri2006/percival2014 0.8855 0.0000 0.0000 0.0934 0.0000 0.0935 0.4109 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.3950 0.1170 0.1537 0.0041 0.2248
oliveira2010/ibt 0.0000 0.0000 0.0000 0.0000 0.4595 0.0000 0.0000 0.0000 1.0000 0.0000 0.0165 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0003 0.0000
percival2014/stem 0.0000 0.4249 0.6642 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.6029 0.9362 0.0000 0.0000 0.0001 0.0000 0.0000 0.0000
scheirer1998/percival2014 0.0000 0.0000 0.0000 0.0000 0.0518 0.0000 0.0000 0.0000 0.0165 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2014/default 0.0000 0.7431 0.9370 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.6029 0.0000 1.0000 0.5505 0.0000 0.0001 0.0006 0.0000 0.0000 0.0000
schreiber2017/ismir2017 0.0000 0.3845 0.6464 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.9362 0.0000 0.5505 1.0000 0.0000 0.0000 0.0001 0.0000 0.0000 0.0000
schreiber2017/mirex2017 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2018/cnn 0.4795 0.0005 0.0003 0.0037 0.0000 0.0184 0.9470 0.3950 0.0000 0.0000 0.0000 0.0001 0.0000 0.0000 1.0000 0.4564 0.0109 0.0000 0.0575
schreiber2018/fcn 0.1613 0.0045 0.0020 0.0005 0.0000 0.0025 0.5422 0.1170 0.0000 0.0001 0.0000 0.0006 0.0001 0.0000 0.4564 1.0000 0.0014 0.0000 0.0107
schreiber2018/ismir2018 0.0898 0.0000 0.0000 0.5650 0.0000 0.7372 0.0465 0.1537 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0109 0.0014 1.0000 0.0486 0.8784
sun2021/default 0.0003 0.0000 0.0000 0.1883 0.0000 0.2623 0.0005 0.0041 0.0003 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0486 1.0000 0.0694
zplane/auftakt_v3 0.2101 0.0000 0.0000 0.5177 0.0000 0.5836 0.0700 0.2248 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0575 0.0107 0.8784 0.0694 1.0000

Table 9: McNemar p-values, using reference annotations 2.0 as groundtruth with Accuracy1 [Gouyon2006]. H0: both estimators disagree with the groundtruth to the same amount. If p<=ɑ, reject H0, i.e. we have a significant difference in the disagreement with the groundtruth. In the table, p-values<0.05 are set in bold.

CSV JSON LATEX PICKLE

Estimator boeck2015/tempodetector2016_default boeck2019/multi_task boeck2019/multi_task_hjdb boeck2020/dar davies2009/mirex_qm_tempotracker echonest/version_3_2_1 gkiokas2012/default klapuri2006/percival2014 oliveira2010/ibt percival2014/stem scheirer1998/percival2014 schreiber2014/default schreiber2017/ismir2017 schreiber2017/mirex2017 schreiber2018/cnn schreiber2018/fcn schreiber2018/ismir2018 sun2021/default zplane/auftakt_v3
boeck2015/tempodetector2016_default 1.0000 0.0000 0.0000 0.0403 0.0000 0.1442 0.4153 0.8843 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.5258 0.1020 0.1213 0.0006 0.3352
boeck2019/multi_task 0.0000 1.0000 0.2624 0.0000 0.0000 0.0000 0.0003 0.0000 0.0000 0.6097 0.0000 1.0000 0.7448 0.0000 0.0001 0.0021 0.0000 0.0000 0.0000
boeck2019/multi_task_hjdb 0.0000 0.2624 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.4683 0.8148 0.0000 0.0000 0.0004 0.0000 0.0000 0.0000
boeck2020/dar 0.0403 0.0000 0.0000 1.0000 0.0000 0.9487 0.0198 0.1568 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0100 0.0007 0.6790 0.1360 0.5160
davies2009/mirex_qm_tempotracker 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.2234 0.0000 0.0937 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
echonest/version_3_2_1 0.1442 0.0000 0.0000 0.9487 0.0000 1.0000 0.0157 0.1636 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0383 0.0029 0.8397 0.2063 0.5818
gkiokas2012/default 0.4153 0.0003 0.0000 0.0198 0.0000 0.0157 1.0000 0.2960 0.0000 0.0000 0.0000 0.0001 0.0000 0.0000 0.8397 0.5365 0.0358 0.0003 0.0789
klapuri2006/percival2014 0.8843 0.0000 0.0000 0.1568 0.0000 0.1636 0.2960 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.4224 0.0727 0.2016 0.0058 0.3823
oliveira2010/ibt 0.0000 0.0000 0.0000 0.0000 0.2234 0.0000 0.0000 0.0000 1.0000 0.0000 0.0155 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0006 0.0000
percival2014/stem 0.0000 0.6097 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.4778 0.8699 0.0000 0.0000 0.0002 0.0000 0.0000 0.0000
scheirer1998/percival2014 0.0000 0.0000 0.0000 0.0000 0.0937 0.0000 0.0000 0.0000 0.0155 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2014/default 0.0000 1.0000 0.4683 0.0000 0.0000 0.0000 0.0001 0.0000 0.0000 0.4778 0.0000 1.0000 0.6135 0.0000 0.0000 0.0012 0.0000 0.0000 0.0000
schreiber2017/ismir2017 0.0000 0.7448 0.8148 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.8699 0.0000 0.6135 1.0000 0.0000 0.0000 0.0002 0.0000 0.0000 0.0000
schreiber2017/mirex2017 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2018/cnn 0.5258 0.0001 0.0000 0.0100 0.0000 0.0383 0.8397 0.4224 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.2685 0.0188 0.0000 0.1133
schreiber2018/fcn 0.1020 0.0021 0.0004 0.0007 0.0000 0.0029 0.5365 0.0727 0.0000 0.0002 0.0000 0.0012 0.0002 0.0000 0.2685 1.0000 0.0010 0.0000 0.0119
schreiber2018/ismir2018 0.1213 0.0000 0.0000 0.6790 0.0000 0.8397 0.0358 0.2016 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0188 0.0010 1.0000 0.0469 0.7555
sun2021/default 0.0006 0.0000 0.0000 0.1360 0.0000 0.2063 0.0003 0.0058 0.0006 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0469 1.0000 0.0483
zplane/auftakt_v3 0.3352 0.0000 0.0000 0.5160 0.0000 0.5818 0.0789 0.3823 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.1133 0.0119 0.7555 0.0483 1.0000

Table 10: McNemar p-values, using reference annotations 1.0 as groundtruth with Accuracy1 [Gouyon2006]. H0: both estimators disagree with the groundtruth to the same amount. If p<=ɑ, reject H0, i.e. we have a significant difference in the disagreement with the groundtruth. In the table, p-values<0.05 are set in bold.

CSV JSON LATEX PICKLE

Estimator boeck2015/tempodetector2016_default boeck2019/multi_task boeck2019/multi_task_hjdb boeck2020/dar davies2009/mirex_qm_tempotracker echonest/version_3_2_1 gkiokas2012/default klapuri2006/percival2014 oliveira2010/ibt percival2014/stem scheirer1998/percival2014 schreiber2014/default schreiber2017/ismir2017 schreiber2017/mirex2017 schreiber2018/cnn schreiber2018/fcn schreiber2018/ismir2018 sun2021/default zplane/auftakt_v3
boeck2015/tempodetector2016_default 1.0000 0.3771 0.2005 0.3833 0.0000 0.0000 0.0005 0.0004 0.0000 0.0436 0.0000 0.0038 0.0038 0.3817 0.1628 0.6177 0.0005 0.0006 0.0000
boeck2019/multi_task 0.3771 1.0000 0.6250 0.0347 0.0001 0.0000 0.0175 0.0076 0.0000 0.3123 0.0000 0.0400 0.0363 1.0000 0.7608 0.8746 0.0222 0.0186 0.0000
boeck2019/multi_task_hjdb 0.2005 0.6250 1.0000 0.0146 0.0002 0.0000 0.0328 0.0139 0.0000 0.4709 0.0000 0.0725 0.0627 1.0000 1.0000 0.6271 0.0402 0.0400 0.0000
boeck2020/dar 0.3833 0.0347 0.0146 1.0000 0.0000 0.0000 0.0001 0.0000 0.0000 0.0066 0.0000 0.0002 0.0001 0.0884 0.0336 0.1628 0.0000 0.0000 0.0000
davies2009/mirex_qm_tempotracker 0.0000 0.0001 0.0002 0.0000 1.0000 0.0001 0.0605 0.1659 0.0001 0.0008 0.0000 0.0320 0.0482 0.0000 0.0001 0.0000 0.0451 0.0429 0.0124
echonest/version_3_2_1 0.0000 0.0000 0.0000 0.0000 0.0001 1.0000 0.0000 0.0000 0.9279 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.1288
gkiokas2012/default 0.0005 0.0175 0.0328 0.0001 0.0605 0.0000 1.0000 0.6350 0.0000 0.2288 0.0000 0.7946 1.0000 0.0331 0.0581 0.0078 0.9005 1.0000 0.0000
klapuri2006/percival2014 0.0004 0.0076 0.0139 0.0000 0.1659 0.0000 0.6350 1.0000 0.0000 0.0722 0.0000 0.3816 0.5557 0.0103 0.0135 0.0010 0.6143 0.5383 0.0000
oliveira2010/ibt 0.0000 0.0000 0.0000 0.0000 0.0001 0.9279 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.1719
percival2014/stem 0.0436 0.3123 0.4709 0.0066 0.0008 0.0000 0.2288 0.0722 0.0000 1.0000 0.0000 0.4101 0.2976 0.4011 0.5224 0.2026 0.2116 0.2806 0.0000
scheirer1998/percival2014 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2014/default 0.0038 0.0400 0.0725 0.0002 0.0320 0.0000 0.7946 0.3816 0.0000 0.4101 0.0000 1.0000 0.8642 0.0488 0.1114 0.0331 0.7838 0.8877 0.0000
schreiber2017/ismir2017 0.0038 0.0363 0.0627 0.0001 0.0482 0.0000 1.0000 0.5557 0.0000 0.2976 0.0000 0.8642 1.0000 0.0009 0.0488 0.0127 1.0000 0.8973 0.0000
schreiber2017/mirex2017 0.3817 1.0000 1.0000 0.0884 0.0000 0.0000 0.0331 0.0103 0.0000 0.4011 0.0000 0.0488 0.0009 1.0000 0.8714 0.7660 0.0241 0.0440 0.0000
schreiber2018/cnn 0.1628 0.7608 1.0000 0.0336 0.0001 0.0000 0.0581 0.0135 0.0000 0.5224 0.0000 0.1114 0.0488 0.8714 1.0000 0.4869 0.0167 0.0541 0.0000
schreiber2018/fcn 0.6177 0.8746 0.6271 0.1628 0.0000 0.0000 0.0078 0.0010 0.0000 0.2026 0.0000 0.0331 0.0127 0.7660 0.4869 1.0000 0.0037 0.0094 0.0000
schreiber2018/ismir2018 0.0005 0.0222 0.0402 0.0000 0.0451 0.0000 0.9005 0.6143 0.0000 0.2116 0.0000 0.7838 1.0000 0.0241 0.0167 0.0037 1.0000 1.0000 0.0000
sun2021/default 0.0006 0.0186 0.0400 0.0000 0.0429 0.0000 1.0000 0.5383 0.0000 0.2806 0.0000 0.8877 0.8973 0.0440 0.0541 0.0094 1.0000 1.0000 0.0000
zplane/auftakt_v3 0.0000 0.0000 0.0000 0.0000 0.0124 0.1288 0.0000 0.0000 0.1719 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000

Table 11: McNemar p-values, using reference annotations GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI as groundtruth with Accuracy2 [Gouyon2006]. H0: both estimators disagree with the groundtruth to the same amount. If p<=ɑ, reject H0, i.e. we have a significant difference in the disagreement with the groundtruth. In the table, p-values<0.05 are set in bold.

CSV JSON LATEX PICKLE

Estimator boeck2015/tempodetector2016_default boeck2019/multi_task boeck2019/multi_task_hjdb boeck2020/dar davies2009/mirex_qm_tempotracker echonest/version_3_2_1 gkiokas2012/default klapuri2006/percival2014 oliveira2010/ibt percival2014/stem scheirer1998/percival2014 schreiber2014/default schreiber2017/ismir2017 schreiber2017/mirex2017 schreiber2018/cnn schreiber2018/fcn schreiber2018/ismir2018 sun2021/default zplane/auftakt_v3
boeck2015/tempodetector2016_default 1.0000 0.7201 0.5847 0.2379 0.0001 0.0000 0.0037 0.0005 0.0000 0.1081 0.0000 0.0045 0.0119 0.6655 0.3915 0.6076 0.0019 0.0002 0.0000
boeck2019/multi_task 0.7201 1.0000 1.0000 0.0931 0.0005 0.0000 0.0241 0.0031 0.0000 0.3123 0.0000 0.0186 0.0363 1.0000 0.7552 1.0000 0.0198 0.0021 0.0000
boeck2019/multi_task_hjdb 0.5847 1.0000 1.0000 0.0639 0.0009 0.0000 0.0328 0.0043 0.0000 0.3916 0.0000 0.0259 0.0479 1.0000 0.8776 1.0000 0.0270 0.0038 0.0000
boeck2020/dar 0.2379 0.0931 0.0639 1.0000 0.0000 0.0000 0.0004 0.0000 0.0000 0.0137 0.0000 0.0002 0.0004 0.1742 0.0652 0.1102 0.0000 0.0000 0.0000
davies2009/mirex_qm_tempotracker 0.0001 0.0005 0.0009 0.0000 1.0000 0.0000 0.1410 0.5557 0.0000 0.0052 0.0000 0.1511 0.1461 0.0004 0.0008 0.0004 0.1544 0.3619 0.0045
echonest/version_3_2_1 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.8557 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.1564
gkiokas2012/default 0.0037 0.0241 0.0328 0.0004 0.1410 0.0000 1.0000 0.4158 0.0000 0.2717 0.0000 0.8937 0.9020 0.0479 0.0759 0.0328 1.0000 0.6198 0.0000
klapuri2006/percival2014 0.0005 0.0031 0.0043 0.0000 0.5557 0.0000 0.4158 1.0000 0.0000 0.0363 0.0000 0.3581 0.4028 0.0043 0.0054 0.0015 0.4424 0.8041 0.0001
oliveira2010/ibt 0.0000 0.0000 0.0000 0.0000 0.0000 0.8557 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.2370
percival2014/stem 0.1081 0.3123 0.3916 0.0137 0.0052 0.0000 0.2717 0.0363 0.0000 1.0000 0.0000 0.2624 0.2976 0.4011 0.5224 0.3916 0.2026 0.0869 0.0000
scheirer1998/percival2014 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2014/default 0.0045 0.0186 0.0259 0.0002 0.1511 0.0000 0.8937 0.3581 0.0000 0.2624 0.0000 1.0000 1.0000 0.0166 0.0488 0.0328 1.0000 0.5901 0.0000
schreiber2017/ismir2017 0.0119 0.0363 0.0479 0.0004 0.1461 0.0000 0.9020 0.4028 0.0000 0.2976 0.0000 1.0000 1.0000 0.0009 0.0436 0.0328 1.0000 0.6085 0.0000
schreiber2017/mirex2017 0.6655 1.0000 1.0000 0.1742 0.0004 0.0000 0.0479 0.0043 0.0000 0.4011 0.0000 0.0166 0.0009 1.0000 0.8679 1.0000 0.0241 0.0086 0.0000
schreiber2018/cnn 0.3915 0.7552 0.8776 0.0652 0.0008 0.0000 0.0759 0.0054 0.0000 0.5224 0.0000 0.0488 0.0436 0.8679 1.0000 0.8601 0.0135 0.0094 0.0000
schreiber2018/fcn 0.6076 1.0000 1.0000 0.1102 0.0004 0.0000 0.0328 0.0015 0.0000 0.3916 0.0000 0.0328 0.0328 1.0000 0.8601 1.0000 0.0161 0.0031 0.0000
schreiber2018/ismir2018 0.0019 0.0198 0.0270 0.0000 0.1544 0.0000 1.0000 0.4424 0.0000 0.2026 0.0000 1.0000 1.0000 0.0241 0.0135 0.0161 1.0000 0.6655 0.0000
sun2021/default 0.0002 0.0021 0.0038 0.0000 0.3619 0.0000 0.6198 0.8041 0.0000 0.0869 0.0000 0.5901 0.6085 0.0086 0.0094 0.0031 0.6655 1.0000 0.0000
zplane/auftakt_v3 0.0000 0.0000 0.0000 0.0000 0.0045 0.1564 0.0000 0.0001 0.2370 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000

Table 12: McNemar p-values, using reference annotations GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI as groundtruth with Accuracy2 [Gouyon2006]. H0: both estimators disagree with the groundtruth to the same amount. If p<=ɑ, reject H0, i.e. we have a significant difference in the disagreement with the groundtruth. In the table, p-values<0.05 are set in bold.

CSV JSON LATEX PICKLE

Estimator boeck2015/tempodetector2016_default boeck2019/multi_task boeck2019/multi_task_hjdb boeck2020/dar davies2009/mirex_qm_tempotracker echonest/version_3_2_1 gkiokas2012/default klapuri2006/percival2014 oliveira2010/ibt percival2014/stem scheirer1998/percival2014 schreiber2014/default schreiber2017/ismir2017 schreiber2017/mirex2017 schreiber2018/cnn schreiber2018/fcn schreiber2018/ismir2018 sun2021/default zplane/auftakt_v3
boeck2015/tempodetector2016_default 1.0000 0.2110 0.4296 0.0639 0.0000 0.0000 0.0114 0.0004 0.0000 0.0541 0.0000 0.0026 0.0300 0.6587 0.5515 0.0175 0.0027 0.0000 0.0000
boeck2019/multi_task 0.2110 1.0000 0.4531 0.0019 0.0000 0.0000 0.2892 0.0160 0.0000 0.5962 0.0000 0.0919 0.3497 0.0984 0.6889 0.2717 0.0980 0.0081 0.0000
boeck2019/multi_task_hjdb 0.4296 0.4531 1.0000 0.0070 0.0000 0.0000 0.1409 0.0052 0.0000 0.3581 0.0000 0.0293 0.1770 0.2116 1.0000 0.1263 0.0363 0.0022 0.0000
boeck2020/dar 0.0639 0.0019 0.0070 1.0000 0.0000 0.0000 0.0001 0.0000 0.0000 0.0004 0.0000 0.0000 0.0001 0.4408 0.0400 0.0000 0.0000 0.0000 0.0000
davies2009/mirex_qm_tempotracker 0.0000 0.0000 0.0000 0.0000 1.0000 0.0121 0.0003 0.0153 0.0060 0.0000 0.0000 0.0020 0.0002 0.0000 0.0000 0.0002 0.0017 0.0214 0.4812
echonest/version_3_2_1 0.0000 0.0000 0.0000 0.0000 0.0121 1.0000 0.0000 0.0000 0.9279 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0532
gkiokas2012/default 0.0114 0.2892 0.1409 0.0001 0.0003 0.0000 1.0000 0.1931 0.0000 0.6936 0.0000 0.7032 1.0000 0.0081 0.0984 0.8937 0.6198 0.1486 0.0000
klapuri2006/percival2014 0.0004 0.0160 0.0052 0.0000 0.0153 0.0000 0.1931 1.0000 0.0000 0.0363 0.0000 0.3284 0.1255 0.0001 0.0019 0.1299 0.3816 0.9075 0.0007
oliveira2010/ibt 0.0000 0.0000 0.0000 0.0000 0.0060 0.9279 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0433
percival2014/stem 0.0541 0.5962 0.3581 0.0004 0.0000 0.0000 0.6936 0.0363 0.0000 1.0000 0.0000 0.3409 0.7911 0.0198 0.2221 0.6778 0.2806 0.0396 0.0000
scheirer1998/percival2014 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2014/default 0.0026 0.0919 0.0293 0.0000 0.0020 0.0000 0.7032 0.3284 0.0000 0.3409 0.0000 1.0000 0.4583 0.0001 0.0213 0.6985 1.0000 0.2976 0.0000
schreiber2017/ismir2017 0.0300 0.3497 0.1770 0.0001 0.0002 0.0000 1.0000 0.1255 0.0000 0.7911 0.0000 0.4583 1.0000 0.0000 0.1114 1.0000 0.4799 0.1034 0.0000
schreiber2017/mirex2017 0.6587 0.0984 0.2116 0.4408 0.0000 0.0000 0.0081 0.0001 0.0000 0.0198 0.0000 0.0001 0.0000 1.0000 0.2221 0.0026 0.0003 0.0000 0.0000
schreiber2018/cnn 0.5515 0.6889 1.0000 0.0400 0.0000 0.0000 0.0984 0.0019 0.0000 0.2221 0.0000 0.0213 0.1114 0.2221 1.0000 0.0725 0.0096 0.0005 0.0000
schreiber2018/fcn 0.0175 0.2717 0.1263 0.0000 0.0002 0.0000 0.8937 0.1299 0.0000 0.6778 0.0000 0.6985 1.0000 0.0026 0.0725 1.0000 0.5831 0.1112 0.0000
schreiber2018/ismir2018 0.0027 0.0980 0.0363 0.0000 0.0017 0.0000 0.6198 0.3816 0.0000 0.2806 0.0000 1.0000 0.4799 0.0003 0.0096 0.5831 1.0000 0.3497 0.0000
sun2021/default 0.0000 0.0081 0.0022 0.0000 0.0214 0.0000 0.1486 0.9075 0.0000 0.0396 0.0000 0.2976 0.1034 0.0000 0.0005 0.1112 0.3497 1.0000 0.0007
zplane/auftakt_v3 0.0000 0.0000 0.0000 0.0000 0.4812 0.0532 0.0000 0.0007 0.0433 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0007 1.0000

Table 13: McNemar p-values, using reference annotations 2.0 as groundtruth with Accuracy2 [Gouyon2006]. H0: both estimators disagree with the groundtruth to the same amount. If p<=ɑ, reject H0, i.e. we have a significant difference in the disagreement with the groundtruth. In the table, p-values<0.05 are set in bold.

CSV JSON LATEX PICKLE

Estimator boeck2015/tempodetector2016_default boeck2019/multi_task boeck2019/multi_task_hjdb boeck2020/dar davies2009/mirex_qm_tempotracker echonest/version_3_2_1 gkiokas2012/default klapuri2006/percival2014 oliveira2010/ibt percival2014/stem scheirer1998/percival2014 schreiber2014/default schreiber2017/ismir2017 schreiber2017/mirex2017 schreiber2018/cnn schreiber2018/fcn schreiber2018/ismir2018 sun2021/default zplane/auftakt_v3
boeck2015/tempodetector2016_default 1.0000 0.2800 0.4296 0.0639 0.0000 0.0000 0.0016 0.0003 0.0000 0.0595 0.0000 0.0005 0.0198 0.8830 0.4514 0.0534 0.0038 0.0002 0.0000
boeck2019/multi_task 0.2800 1.0000 0.6875 0.0039 0.0000 0.0000 0.0925 0.0071 0.0000 0.5115 0.0000 0.0213 0.2203 0.2116 0.8937 0.4270 0.0869 0.0111 0.0000
boeck2019/multi_task_hjdb 0.4296 0.6875 1.0000 0.0070 0.0000 0.0000 0.0479 0.0029 0.0000 0.3581 0.0000 0.0079 0.1337 0.3317 0.8937 0.2806 0.0440 0.0044 0.0000
boeck2020/dar 0.0639 0.0039 0.0070 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0005 0.0000 0.0000 0.0000 0.2800 0.0195 0.0002 0.0000 0.0000 0.0000
davies2009/mirex_qm_tempotracker 0.0000 0.0000 0.0000 0.0000 1.0000 0.0184 0.0007 0.0142 0.0265 0.0000 0.0000 0.0034 0.0002 0.0000 0.0000 0.0000 0.0008 0.0104 0.5228
echonest/version_3_2_1 0.0000 0.0000 0.0000 0.0000 0.0184 1.0000 0.0000 0.0000 0.7842 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0622
gkiokas2012/default 0.0016 0.0925 0.0479 0.0000 0.0007 0.0000 1.0000 0.3019 0.0000 0.3409 0.0000 0.7946 0.7077 0.0032 0.0365 0.4270 0.8937 0.4635 0.0000
klapuri2006/percival2014 0.0003 0.0071 0.0029 0.0000 0.0142 0.0000 0.3019 1.0000 0.0000 0.0222 0.0000 0.4424 0.1149 0.0001 0.0011 0.0430 0.2370 0.8149 0.0007
oliveira2010/ibt 0.0000 0.0000 0.0000 0.0000 0.0265 0.7842 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.1219
percival2014/stem 0.0595 0.5115 0.3581 0.0005 0.0000 0.0000 0.3409 0.0222 0.0000 1.0000 0.0000 0.1608 0.6835 0.0365 0.2800 1.0000 0.3020 0.0722 0.0000
scheirer1998/percival2014 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2014/default 0.0005 0.0213 0.0079 0.0000 0.0034 0.0000 0.7946 0.4424 0.0000 0.1608 0.0000 1.0000 0.2649 0.0000 0.0079 0.2288 0.7754 0.6936 0.0000
schreiber2017/ismir2017 0.0198 0.2203 0.1337 0.0000 0.0002 0.0000 0.7077 0.1149 0.0000 0.6835 0.0000 0.2649 1.0000 0.0000 0.0961 0.7798 0.6718 0.2000 0.0000
schreiber2017/mirex2017 0.8830 0.2116 0.3317 0.2800 0.0000 0.0000 0.0032 0.0001 0.0000 0.0365 0.0000 0.0000 0.0000 1.0000 0.2682 0.0241 0.0015 0.0002 0.0000
schreiber2018/cnn 0.4514 0.8937 0.8937 0.0195 0.0000 0.0000 0.0365 0.0011 0.0000 0.2800 0.0000 0.0079 0.0961 0.2682 1.0000 0.2110 0.0195 0.0018 0.0000
schreiber2018/fcn 0.0534 0.4270 0.2806 0.0002 0.0000 0.0000 0.4270 0.0430 0.0000 1.0000 0.0000 0.2288 0.7798 0.0241 0.2110 1.0000 0.3916 0.0759 0.0000
schreiber2018/ismir2018 0.0038 0.0869 0.0440 0.0000 0.0008 0.0000 0.8937 0.2370 0.0000 0.3020 0.0000 0.7754 0.6718 0.0015 0.0195 0.3916 1.0000 0.4188 0.0000
sun2021/default 0.0002 0.0111 0.0044 0.0000 0.0104 0.0000 0.4635 0.8149 0.0000 0.0722 0.0000 0.6936 0.2000 0.0002 0.0018 0.0759 0.4188 1.0000 0.0003
zplane/auftakt_v3 0.0000 0.0000 0.0000 0.0000 0.5228 0.0622 0.0000 0.0007 0.1219 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0003 1.0000

Table 14: McNemar p-values, using reference annotations 1.0 as groundtruth with Accuracy2 [Gouyon2006]. H0: both estimators disagree with the groundtruth to the same amount. If p<=ɑ, reject H0, i.e. we have a significant difference in the disagreement with the groundtruth. In the table, p-values<0.05 are set in bold.

CSV JSON LATEX PICKLE

Accuracy1 on cvar-Subsets

How well does an estimator perform, when only taking tracks into account that have a cvar-value of less than τ, i.e., have a more or less stable beat?

Accuracy1 on cvar-Subsets for 1.0 based on cvar-Values from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28

Figure 14: Mean Accuracy1 compared to version 1.0 for tracks with cvar < τ based on beat annotations from 1.0.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy1 on cvar-Subsets for 2.0 based on cvar-Values from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28

Figure 15: Mean Accuracy1 compared to version 2.0 for tracks with cvar < τ based on beat annotations from 2.0.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy1 on cvar-Subsets for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI based on cvar-Values from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28

Figure 16: Mean Accuracy1 compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI for tracks with cvar < τ based on beat annotations from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy1 on cvar-Subsets for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI based on cvar-Values from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28

Figure 17: Mean Accuracy1 compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI for tracks with cvar < τ based on beat annotations from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy2 on cvar-Subsets

How well does an estimator perform, when only taking tracks into account that have a cvar-value of less than τ, i.e., have a more or less stable beat?

Accuracy2 on cvar-Subsets for 1.0 based on cvar-Values from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28

Figure 18: Mean Accuracy2 compared to version 1.0 for tracks with cvar < τ based on beat annotations from 1.0.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy2 on cvar-Subsets for 2.0 based on cvar-Values from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28

Figure 19: Mean Accuracy2 compared to version 2.0 for tracks with cvar < τ based on beat annotations from 2.0.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy2 on cvar-Subsets for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI based on cvar-Values from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28

Figure 20: Mean Accuracy2 compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI for tracks with cvar < τ based on beat annotations from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy2 on cvar-Subsets for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI based on cvar-Values from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28

Figure 21: Mean Accuracy2 compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI for tracks with cvar < τ based on beat annotations from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy1 on Tempo-Subsets

How well does an estimator perform, when only taking a subset of the reference annotations into account? The graphs show mean Accuracy1 for reference subsets with tempi in [T-10,T+10] BPM. Note that the graphs do not show confidence intervals and that some values may be based on very few estimates.

Accuracy1 on Tempo-Subsets for 1.0

Figure 22: Mean Accuracy1 for estimates compared to version 1.0 for tempo intervals around T.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy1 on Tempo-Subsets for 2.0

Figure 23: Mean Accuracy1 for estimates compared to version 2.0 for tempo intervals around T.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy1 on Tempo-Subsets for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI

Figure 24: Mean Accuracy1 for estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI for tempo intervals around T.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy1 on Tempo-Subsets for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI

Figure 25: Mean Accuracy1 for estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI for tempo intervals around T.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy2 on Tempo-Subsets

How well does an estimator perform, when only taking a subset of the reference annotations into account? The graphs show mean Accuracy2 for reference subsets with tempi in [T-10,T+10] BPM. Note that the graphs do not show confidence intervals and that some values may be based on very few estimates.

Accuracy2 on Tempo-Subsets for 1.0

Figure 26: Mean Accuracy2 for estimates compared to version 1.0 for tempo intervals around T.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy2 on Tempo-Subsets for 2.0

Figure 27: Mean Accuracy2 for estimates compared to version 2.0 for tempo intervals around T.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy2 on Tempo-Subsets for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI

Figure 28: Mean Accuracy2 for estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI for tempo intervals around T.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy2 on Tempo-Subsets for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI

Figure 29: Mean Accuracy2 for estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI for tempo intervals around T.

CSV JSON LATEX PICKLE SVG PDF PNG

Estimated Accuracy1 for Tempo

When fitting a generalized additive model (GAM) to Accuracy1-values and a ground truth, what Accuracy1 can we expect with confidence?

Estimated Accuracy1 for Tempo for 1.0

Predictions of GAMs trained on Accuracy1 for estimates for reference 1.0.

Figure 30: Accuracy1 predictions of a generalized additive model (GAM) fit to Accuracy1 results for 1.0. The 95% confidence interval around the prediction is shaded in gray.

CSV JSON LATEX PICKLE SVG PDF PNG

Estimated Accuracy1 for Tempo for 2.0

Predictions of GAMs trained on Accuracy1 for estimates for reference 2.0.

Figure 31: Accuracy1 predictions of a generalized additive model (GAM) fit to Accuracy1 results for 2.0. The 95% confidence interval around the prediction is shaded in gray.

CSV JSON LATEX PICKLE SVG PDF PNG

Estimated Accuracy1 for Tempo for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI

Predictions of GAMs trained on Accuracy1 for estimates for reference GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI.

Figure 32: Accuracy1 predictions of a generalized additive model (GAM) fit to Accuracy1 results for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI. The 95% confidence interval around the prediction is shaded in gray.

CSV JSON LATEX PICKLE SVG PDF PNG

Estimated Accuracy1 for Tempo for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI

Predictions of GAMs trained on Accuracy1 for estimates for reference GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI.

Figure 33: Accuracy1 predictions of a generalized additive model (GAM) fit to Accuracy1 results for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI. The 95% confidence interval around the prediction is shaded in gray.

CSV JSON LATEX PICKLE SVG PDF PNG

Estimated Accuracy2 for Tempo

When fitting a generalized additive model (GAM) to Accuracy2-values and a ground truth, what Accuracy2 can we expect with confidence?

Estimated Accuracy2 for Tempo for 1.0

Predictions of GAMs trained on Accuracy2 for estimates for reference 1.0.

Figure 34: Accuracy2 predictions of a generalized additive model (GAM) fit to Accuracy2 results for 1.0. The 95% confidence interval around the prediction is shaded in gray.

CSV JSON LATEX PICKLE SVG PDF PNG

Estimated Accuracy2 for Tempo for 2.0

Predictions of GAMs trained on Accuracy2 for estimates for reference 2.0.

Figure 35: Accuracy2 predictions of a generalized additive model (GAM) fit to Accuracy2 results for 2.0. The 95% confidence interval around the prediction is shaded in gray.

CSV JSON LATEX PICKLE SVG PDF PNG

Estimated Accuracy2 for Tempo for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI

Predictions of GAMs trained on Accuracy2 for estimates for reference GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI.

Figure 36: Accuracy2 predictions of a generalized additive model (GAM) fit to Accuracy2 results for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI. The 95% confidence interval around the prediction is shaded in gray.

CSV JSON LATEX PICKLE SVG PDF PNG

Estimated Accuracy2 for Tempo for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI

Predictions of GAMs trained on Accuracy2 for estimates for reference GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI.

Figure 37: Accuracy2 predictions of a generalized additive model (GAM) fit to Accuracy2 results for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI. The 95% confidence interval around the prediction is shaded in gray.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy1 for ‘tag_open’ Tags

How well does an estimator perform, when only taking tracks into account that are tagged with some kind of label? Note that some values may be based on very few estimates.

Accuracy1 for ‘tag_open’ Tags for 1.0

Figure 38: Mean Accuracy1 of estimates compared to version 1.0 depending on tag from namespace ‘tag_open’.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy1 for ‘tag_open’ Tags for 2.0

Figure 39: Mean Accuracy1 of estimates compared to version 2.0 depending on tag from namespace ‘tag_open’.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy1 for ‘tag_open’ Tags for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI

Figure 40: Mean Accuracy1 of estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI depending on tag from namespace ‘tag_open’.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy1 for ‘tag_open’ Tags for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI

Figure 41: Mean Accuracy1 of estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI depending on tag from namespace ‘tag_open’.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy1 for ‘tag_gtzan’ Tags

How well does an estimator perform, when only taking tracks into account that are tagged with some kind of label? Note that some values may be based on very few estimates.

Accuracy1 for ‘tag_gtzan’ Tags for 1.0

Figure 42: Mean Accuracy1 of estimates compared to version 1.0 depending on tag from namespace ‘tag_gtzan’.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy1 for ‘tag_gtzan’ Tags for 2.0

Figure 43: Mean Accuracy1 of estimates compared to version 2.0 depending on tag from namespace ‘tag_gtzan’.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy1 for ‘tag_gtzan’ Tags for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI

Figure 44: Mean Accuracy1 of estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI depending on tag from namespace ‘tag_gtzan’.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy1 for ‘tag_gtzan’ Tags for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI

Figure 45: Mean Accuracy1 of estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI depending on tag from namespace ‘tag_gtzan’.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy2 for ‘tag_open’ Tags

How well does an estimator perform, when only taking tracks into account that are tagged with some kind of label? Note that some values may be based on very few estimates.

Accuracy2 for ‘tag_open’ Tags for 1.0

Figure 46: Mean Accuracy2 of estimates compared to version 1.0 depending on tag from namespace ‘tag_open’.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy2 for ‘tag_open’ Tags for 2.0

Figure 47: Mean Accuracy2 of estimates compared to version 2.0 depending on tag from namespace ‘tag_open’.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy2 for ‘tag_open’ Tags for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI

Figure 48: Mean Accuracy2 of estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI depending on tag from namespace ‘tag_open’.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy2 for ‘tag_open’ Tags for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI

Figure 49: Mean Accuracy2 of estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI depending on tag from namespace ‘tag_open’.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy2 for ‘tag_gtzan’ Tags

How well does an estimator perform, when only taking tracks into account that are tagged with some kind of label? Note that some values may be based on very few estimates.

Accuracy2 for ‘tag_gtzan’ Tags for 1.0

Figure 50: Mean Accuracy2 of estimates compared to version 1.0 depending on tag from namespace ‘tag_gtzan’.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy2 for ‘tag_gtzan’ Tags for 2.0

Figure 51: Mean Accuracy2 of estimates compared to version 2.0 depending on tag from namespace ‘tag_gtzan’.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy2 for ‘tag_gtzan’ Tags for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI

Figure 52: Mean Accuracy2 of estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI depending on tag from namespace ‘tag_gtzan’.

CSV JSON LATEX PICKLE SVG PDF PNG

Accuracy2 for ‘tag_gtzan’ Tags for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI

Figure 53: Mean Accuracy2 of estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI depending on tag from namespace ‘tag_gtzan’.

CSV JSON LATEX PICKLE SVG PDF PNG

OE1 and OE2

OE1 is defined as octave error between an estimate E and a reference value R.This means that the most common errors—by a factor of 2 or ½—have the same magnitude, namely 1: OE2(E) = log2(E/R).

OE2 is the signed OE1 corresponding to the minimum absolute OE1 allowing the octaveerrors 2, 3, 1/2, and 1/3: OE2(E) = arg minx(|x|) with x ∈ {OE1(E), OE1(2E), OE1(3E), OE1(½E), OE1(⅓E)}

Mean OE1/OE2 Results for 1.0

Estimator OE1_MEAN OE1_STDEV OE2_MEAN OE2_STDEV
schreiber2017/mirex2017 0.0590 0.3069 -0.0021 0.0487
schreiber2017/ismir2017 0.1440 0.4068 -0.0048 0.0797
percival2014/stem 0.1399 0.4119 -0.0008 0.0728
schreiber2014/default 0.1258 0.4130 -0.0107 0.0895
boeck2019/multi_task 0.0764 0.4306 -0.0013 0.0713
boeck2019/multi_task_hjdb 0.0522 0.4308 -0.0023 0.0712
schreiber2018/fcn 0.2068 0.4507 -0.0047 0.0698
schreiber2018/cnn 0.2473 0.4514 -0.0043 0.0741
echonest/version_3_2_1 0.1468 0.4524 -0.0091 0.1115
klapuri2006/percival2014 0.2610 0.4597 -0.0076 0.0840
schreiber2018/ismir2018 0.2691 0.4657 -0.0050 0.0900
sun2021/default 0.2753 0.4667 -0.0115 0.0828
zplane/auftakt_v3 0.2372 0.4738 -0.0119 0.1109
boeck2020/dar 0.2857 0.4770 0.0004 0.0589
oliveira2010/ibt 0.3338 0.4843 -0.0121 0.1021
gkiokas2012/default 0.1870 0.4878 -0.0032 0.0863
boeck2015/tempodetector2016_default 0.2759 0.4898 0.0006 0.0684
davies2009/mirex_qm_tempotracker 0.3979 0.5083 0.0173 0.0769
scheirer1998/percival2014 0.1383 0.5280 0.0207 0.1605

Table 15: Mean OE1/OE2 for estimates compared to version 1.0 ordered by standard deviation.

CSV JSON LATEX PICKLE

Raw data OE1: CSV JSON LATEX PICKLE

Raw data OE2: CSV JSON LATEX PICKLE

OE1 distribution for 1.0

Figure 54: OE1 for estimates compared to version 1.0. Shown are the mean OE1 and an empirical distribution of the sample, using kernel density estimation (KDE).

CSV JSON LATEX PICKLE SVG PDF PNG

OE2 distribution for 1.0

Figure 55: OE2 for estimates compared to version 1.0. Shown are the mean OE2 and an empirical distribution of the sample, using kernel density estimation (KDE).

CSV JSON LATEX PICKLE SVG PDF PNG

Mean OE1/OE2 Results for 2.0

Estimator OE1_MEAN OE1_STDEV OE2_MEAN OE2_STDEV
schreiber2017/mirex2017 0.0472 0.3074 -0.0034 0.0434
schreiber2017/ismir2017 0.1332 0.4073 -0.0067 0.0758
percival2014/stem 0.1292 0.4110 0.0003 0.0739
schreiber2014/default 0.1141 0.4122 -0.0126 0.0860
boeck2019/multi_task 0.0658 0.4364 -0.0019 0.0688
boeck2019/multi_task_hjdb 0.0415 0.4407 -0.0029 0.0687
schreiber2018/fcn 0.1951 0.4506 -0.0056 0.0699
schreiber2018/cnn 0.2356 0.4528 -0.0041 0.0743
klapuri2006/percival2014 0.2500 0.4543 -0.0077 0.0832
echonest/version_3_2_1 0.1366 0.4605 -0.0095 0.1093
schreiber2018/ismir2018 0.2573 0.4676 -0.0064 0.0871
zplane/auftakt_v3 0.2256 0.4685 -0.0120 0.1105
sun2021/default 0.2646 0.4694 -0.0146 0.0788
oliveira2010/ibt 0.3223 0.4827 -0.0118 0.1023
boeck2020/dar 0.2750 0.4832 -0.0013 0.0521
boeck2015/tempodetector2016_default 0.2643 0.4878 -0.0007 0.0612
gkiokas2012/default 0.1758 0.4884 -0.0028 0.0819
davies2009/mirex_qm_tempotracker 0.3867 0.5042 0.0169 0.0741
scheirer1998/percival2014 0.1263 0.5228 0.0241 0.1601

Table 16: Mean OE1/OE2 for estimates compared to version 2.0 ordered by standard deviation.

CSV JSON LATEX PICKLE

Raw data OE1: CSV JSON LATEX PICKLE

Raw data OE2: CSV JSON LATEX PICKLE

OE1 distribution for 2.0

Figure 56: OE1 for estimates compared to version 2.0. Shown are the mean OE1 and an empirical distribution of the sample, using kernel density estimation (KDE).

CSV JSON LATEX PICKLE SVG PDF PNG

OE2 distribution for 2.0

Figure 57: OE2 for estimates compared to version 2.0. Shown are the mean OE2 and an empirical distribution of the sample, using kernel density estimation (KDE).

CSV JSON LATEX PICKLE SVG PDF PNG

Mean OE1/OE2 Results for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI

Estimator OE1_MEAN OE1_STDEV OE2_MEAN OE2_STDEV
boeck2020/dar -0.0325 0.3644 0.0020 0.0638
sun2021/default -0.0429 0.3940 -0.0116 0.0818
schreiber2018/ismir2018 -0.0497 0.4310 -0.0062 0.0918
schreiber2018/cnn -0.0714 0.4512 -0.0016 0.0768
schreiber2018/fcn -0.1120 0.4611 0.0014 0.0679
boeck2015/tempodetector2016_default -0.0429 0.4656 0.0013 0.0682
boeck2019/multi_task -0.2418 0.4717 0.0018 0.0739
schreiber2017/ismir2017 -0.1748 0.4838 -0.0024 0.0848
schreiber2014/default -0.1929 0.4906 -0.0099 0.0838
boeck2019/multi_task_hjdb -0.2660 0.4917 -0.0006 0.0751
echonest/version_3_2_1 -0.1713 0.5000 -0.0044 0.1110
schreiber2017/mirex2017 -0.2597 0.5001 -0.0010 0.0696
oliveira2010/ibt 0.0151 0.5025 -0.0114 0.0980
klapuri2006/percival2014 -0.0578 0.5081 -0.0037 0.0818
percival2014/stem -0.1788 0.5138 -0.0006 0.0732
zplane/auftakt_v3 -0.0816 0.5194 -0.0101 0.1099
davies2009/mirex_qm_tempotracker 0.0792 0.5219 0.0198 0.0746
gkiokas2012/default -0.1318 0.5595 -0.0001 0.0820
scheirer1998/percival2014 -0.1772 0.5888 0.0223 0.1565

Table 17: Mean OE1/OE2 for estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI ordered by standard deviation.

CSV JSON LATEX PICKLE

Raw data OE1: CSV JSON LATEX PICKLE

Raw data OE2: CSV JSON LATEX PICKLE

OE1 distribution for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI

Figure 58: OE1 for estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI. Shown are the mean OE1 and an empirical distribution of the sample, using kernel density estimation (KDE).

CSV JSON LATEX PICKLE SVG PDF PNG

OE2 distribution for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI

Figure 59: OE2 for estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI. Shown are the mean OE2 and an empirical distribution of the sample, using kernel density estimation (KDE).

CSV JSON LATEX PICKLE SVG PDF PNG

Mean OE1/OE2 Results for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI

Estimator OE1_MEAN OE1_STDEV OE2_MEAN OE2_STDEV
boeck2020/dar -0.0320 0.3641 0.0036 0.0649
sun2021/default -0.0423 0.3939 -0.0111 0.0812
schreiber2018/ismir2018 -0.0491 0.4310 -0.0057 0.0923
schreiber2018/cnn -0.0709 0.4514 -0.0020 0.0768
schreiber2018/fcn -0.1115 0.4610 0.0020 0.0687
boeck2015/tempodetector2016_default -0.0423 0.4660 0.0018 0.0688
boeck2019/multi_task -0.2412 0.4712 0.0024 0.0748
schreiber2017/ismir2017 -0.1743 0.4837 -0.0018 0.0852
schreiber2014/default -0.1924 0.4904 -0.0084 0.0833
boeck2019/multi_task_hjdb -0.2654 0.4913 -0.0001 0.0762
echonest/version_3_2_1 -0.1707 0.4999 -0.0038 0.1113
schreiber2017/mirex2017 -0.2592 0.5000 -0.0004 0.0701
oliveira2010/ibt 0.0156 0.5026 -0.0103 0.0972
klapuri2006/percival2014 -0.0572 0.5082 -0.0042 0.0818
percival2014/stem -0.1783 0.5137 -0.0001 0.0733
zplane/auftakt_v3 -0.0810 0.5192 -0.0089 0.1099
davies2009/mirex_qm_tempotracker 0.0797 0.5222 0.0194 0.0742
gkiokas2012/default -0.1313 0.5600 0.0010 0.0821
scheirer1998/percival2014 -0.1766 0.5890 0.0228 0.1566

Table 18: Mean OE1/OE2 for estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI ordered by standard deviation.

CSV JSON LATEX PICKLE

Raw data OE1: CSV JSON LATEX PICKLE

Raw data OE2: CSV JSON LATEX PICKLE

OE1 distribution for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI

Figure 60: OE1 for estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI. Shown are the mean OE1 and an empirical distribution of the sample, using kernel density estimation (KDE).

CSV JSON LATEX PICKLE SVG PDF PNG

OE2 distribution for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI

Figure 61: OE2 for estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI. Shown are the mean OE2 and an empirical distribution of the sample, using kernel density estimation (KDE).

CSV JSON LATEX PICKLE SVG PDF PNG

Significance of Differences

Estimator boeck2015/tempodetector2016_default boeck2019/multi_task boeck2019/multi_task_hjdb boeck2020/dar davies2009/mirex_qm_tempotracker echonest/version_3_2_1 gkiokas2012/default klapuri2006/percival2014 oliveira2010/ibt percival2014/stem scheirer1998/percival2014 schreiber2014/default schreiber2017/ismir2017 schreiber2017/mirex2017 schreiber2018/cnn schreiber2018/fcn schreiber2018/ismir2018 sun2021/default zplane/auftakt_v3
boeck2015/tempodetector2016_default 1.0000 0.0000 0.0000 0.4092 0.0000 0.0000 0.0000 0.2873 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0256 0.0000 0.5785 0.9760 0.0049
boeck2019/multi_task 0.0000 1.0000 0.0012 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0002 0.0000 0.0000 0.1614 0.0000 0.0000 0.0000 0.0000 0.0000
boeck2019/multi_task_hjdb 0.0000 0.0012 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.6815 0.0000 0.0000 0.0000 0.0000 0.0000
boeck2020/dar 0.4092 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0971 0.0016 0.0000 0.0000 0.0000 0.0000 0.0000 0.0018 0.0000 0.1298 0.3418 0.0014
davies2009/mirex_qm_tempotracker 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
echonest/version_3_2_1 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0072 0.0000 0.0000 0.6144 0.6945 0.0926 0.8350 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
gkiokas2012/default 0.0000 0.0000 0.0000 0.0000 0.0000 0.0072 1.0000 0.0000 0.0000 0.0006 0.0051 0.0000 0.0025 0.0000 0.0001 0.2040 0.0000 0.0000 0.0005
klapuri2006/percival2014 0.2873 0.0000 0.0000 0.0971 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.3244 0.0001 0.4935 0.3092 0.0241
oliveira2010/ibt 0.0000 0.0000 0.0000 0.0016 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
percival2014/stem 0.0000 0.0000 0.0000 0.0000 0.0000 0.6144 0.0006 0.0000 0.0000 1.0000 0.9737 0.1875 0.7277 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
scheirer1998/percival2014 0.0000 0.0002 0.0000 0.0000 0.0000 0.6945 0.0051 0.0000 0.0000 0.9737 1.0000 0.3727 0.8062 0.0000 0.0000 0.0001 0.0000 0.0000 0.0000
schreiber2014/default 0.0000 0.0000 0.0000 0.0000 0.0000 0.0926 0.0000 0.0000 0.0000 0.1875 0.3727 1.0000 0.0687 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2017/ismir2017 0.0000 0.0000 0.0000 0.0000 0.0000 0.8350 0.0025 0.0000 0.0000 0.7277 0.8062 0.0687 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2017/mirex2017 0.0000 0.1614 0.6815 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2018/cnn 0.0256 0.0000 0.0000 0.0018 0.0000 0.0000 0.0001 0.3244 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0007 0.0569 0.0145 0.4600
schreiber2018/fcn 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.2040 0.0001 0.0000 0.0000 0.0001 0.0000 0.0000 0.0000 0.0007 1.0000 0.0000 0.0000 0.0352
schreiber2018/ismir2018 0.5785 0.0000 0.0000 0.1298 0.0000 0.0000 0.0000 0.4935 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0569 0.0000 1.0000 0.5124 0.0113
sun2021/default 0.9760 0.0000 0.0000 0.3418 0.0000 0.0000 0.0000 0.3092 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0145 0.0000 0.5124 1.0000 0.0064
zplane/auftakt_v3 0.0049 0.0000 0.0000 0.0014 0.0000 0.0000 0.0005 0.0241 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.4600 0.0352 0.0113 0.0064 1.0000

Table 19: Paired t-test p-values, using reference annotations GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI as groundtruth with OE1. H0: the true mean difference between paired samples is zero. If p<=ɑ, reject H0, i.e. we have a significant difference between estimates from the two algorithms. In the table, p-values<0.05 are set in bold.

CSV JSON LATEX PICKLE

Estimator boeck2015/tempodetector2016_default boeck2019/multi_task boeck2019/multi_task_hjdb boeck2020/dar davies2009/mirex_qm_tempotracker echonest/version_3_2_1 gkiokas2012/default klapuri2006/percival2014 oliveira2010/ibt percival2014/stem scheirer1998/percival2014 schreiber2014/default schreiber2017/ismir2017 schreiber2017/mirex2017 schreiber2018/cnn schreiber2018/fcn schreiber2018/ismir2018 sun2021/default zplane/auftakt_v3
boeck2015/tempodetector2016_default 1.0000 0.0000 0.0000 0.4092 0.0000 0.0000 0.0000 0.2873 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0256 0.0000 0.5785 0.9760 0.0049
boeck2019/multi_task 0.0000 1.0000 0.0012 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0002 0.0000 0.0000 0.1614 0.0000 0.0000 0.0000 0.0000 0.0000
boeck2019/multi_task_hjdb 0.0000 0.0012 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.6815 0.0000 0.0000 0.0000 0.0000 0.0000
boeck2020/dar 0.4092 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0971 0.0016 0.0000 0.0000 0.0000 0.0000 0.0000 0.0018 0.0000 0.1298 0.3418 0.0014
davies2009/mirex_qm_tempotracker 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
echonest/version_3_2_1 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0072 0.0000 0.0000 0.6144 0.6945 0.0926 0.8350 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
gkiokas2012/default 0.0000 0.0000 0.0000 0.0000 0.0000 0.0072 1.0000 0.0000 0.0000 0.0006 0.0051 0.0000 0.0025 0.0000 0.0001 0.2040 0.0000 0.0000 0.0005
klapuri2006/percival2014 0.2873 0.0000 0.0000 0.0971 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.3244 0.0001 0.4935 0.3092 0.0241
oliveira2010/ibt 0.0000 0.0000 0.0000 0.0016 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
percival2014/stem 0.0000 0.0000 0.0000 0.0000 0.0000 0.6144 0.0006 0.0000 0.0000 1.0000 0.9737 0.1875 0.7277 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
scheirer1998/percival2014 0.0000 0.0002 0.0000 0.0000 0.0000 0.6945 0.0051 0.0000 0.0000 0.9737 1.0000 0.3727 0.8062 0.0000 0.0000 0.0001 0.0000 0.0000 0.0000
schreiber2014/default 0.0000 0.0000 0.0000 0.0000 0.0000 0.0926 0.0000 0.0000 0.0000 0.1875 0.3727 1.0000 0.0687 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2017/ismir2017 0.0000 0.0000 0.0000 0.0000 0.0000 0.8350 0.0025 0.0000 0.0000 0.7277 0.8062 0.0687 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2017/mirex2017 0.0000 0.1614 0.6815 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2018/cnn 0.0256 0.0000 0.0000 0.0018 0.0000 0.0000 0.0001 0.3244 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0007 0.0569 0.0145 0.4600
schreiber2018/fcn 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.2040 0.0001 0.0000 0.0000 0.0001 0.0000 0.0000 0.0000 0.0007 1.0000 0.0000 0.0000 0.0352
schreiber2018/ismir2018 0.5785 0.0000 0.0000 0.1298 0.0000 0.0000 0.0000 0.4935 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0569 0.0000 1.0000 0.5124 0.0113
sun2021/default 0.9760 0.0000 0.0000 0.3418 0.0000 0.0000 0.0000 0.3092 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0145 0.0000 0.5124 1.0000 0.0064
zplane/auftakt_v3 0.0049 0.0000 0.0000 0.0014 0.0000 0.0000 0.0005 0.0241 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.4600 0.0352 0.0113 0.0064 1.0000

Table 20: Paired t-test p-values, using reference annotations GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI as groundtruth with OE1. H0: the true mean difference between paired samples is zero. If p<=ɑ, reject H0, i.e. we have a significant difference between estimates from the two algorithms. In the table, p-values<0.05 are set in bold.

CSV JSON LATEX PICKLE

Estimator boeck2015/tempodetector2016_default boeck2019/multi_task boeck2019/multi_task_hjdb boeck2020/dar davies2009/mirex_qm_tempotracker echonest/version_3_2_1 gkiokas2012/default klapuri2006/percival2014 oliveira2010/ibt percival2014/stem scheirer1998/percival2014 schreiber2014/default schreiber2017/ismir2017 schreiber2017/mirex2017 schreiber2018/cnn schreiber2018/fcn schreiber2018/ismir2018 sun2021/default zplane/auftakt_v3
boeck2015/tempodetector2016_default 1.0000 0.0000 0.0000 0.4092 0.0000 0.0000 0.0000 0.3064 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0252 0.0000 0.5706 0.9760 0.0049
boeck2019/multi_task 0.0000 1.0000 0.0012 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0002 0.0000 0.0000 0.1614 0.0000 0.0000 0.0000 0.0000 0.0000
boeck2019/multi_task_hjdb 0.0000 0.0012 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.6815 0.0000 0.0000 0.0000 0.0000 0.0000
boeck2020/dar 0.4092 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0971 0.0016 0.0000 0.0000 0.0000 0.0000 0.0000 0.0018 0.0000 0.1298 0.3418 0.0014
davies2009/mirex_qm_tempotracker 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
echonest/version_3_2_1 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0085 0.0000 0.0000 0.5867 0.6238 0.0688 0.8030 0.0000 0.0000 0.0001 0.0000 0.0000 0.0000
gkiokas2012/default 0.0000 0.0000 0.0000 0.0000 0.0000 0.0085 1.0000 0.0000 0.0000 0.0006 0.0045 0.0000 0.0028 0.0000 0.0001 0.2172 0.0000 0.0000 0.0006
klapuri2006/percival2014 0.3064 0.0000 0.0000 0.0971 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.3007 0.0001 0.5328 0.3092 0.0206
oliveira2010/ibt 0.0000 0.0000 0.0000 0.0016 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
percival2014/stem 0.0000 0.0000 0.0000 0.0000 0.0000 0.5867 0.0006 0.0000 0.0000 1.0000 0.9174 0.1562 0.7277 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
scheirer1998/percival2014 0.0000 0.0002 0.0000 0.0000 0.0000 0.6238 0.0045 0.0000 0.0000 0.9174 1.0000 0.3731 0.7540 0.0000 0.0000 0.0001 0.0000 0.0000 0.0000
schreiber2014/default 0.0000 0.0000 0.0000 0.0000 0.0000 0.0688 0.0000 0.0000 0.0000 0.1562 0.3731 1.0000 0.0535 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2017/ismir2017 0.0000 0.0000 0.0000 0.0000 0.0000 0.8030 0.0028 0.0000 0.0000 0.7277 0.7540 0.0535 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2017/mirex2017 0.0000 0.1614 0.6815 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2018/cnn 0.0252 0.0000 0.0000 0.0018 0.0000 0.0000 0.0001 0.3007 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0008 0.0577 0.0145 0.4637
schreiber2018/fcn 0.0000 0.0000 0.0000 0.0000 0.0000 0.0001 0.2172 0.0001 0.0000 0.0000 0.0001 0.0000 0.0000 0.0000 0.0008 1.0000 0.0000 0.0000 0.0352
schreiber2018/ismir2018 0.5706 0.0000 0.0000 0.1298 0.0000 0.0000 0.0000 0.5328 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0577 0.0000 1.0000 0.5124 0.0117
sun2021/default 0.9760 0.0000 0.0000 0.3418 0.0000 0.0000 0.0000 0.3092 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0145 0.0000 0.5124 1.0000 0.0064
zplane/auftakt_v3 0.0049 0.0000 0.0000 0.0014 0.0000 0.0000 0.0006 0.0206 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.4637 0.0352 0.0117 0.0064 1.0000

Table 21: Paired t-test p-values, using reference annotations 2.0 as groundtruth with OE1. H0: the true mean difference between paired samples is zero. If p<=ɑ, reject H0, i.e. we have a significant difference between estimates from the two algorithms. In the table, p-values<0.05 are set in bold.

CSV JSON LATEX PICKLE

Estimator boeck2015/tempodetector2016_default boeck2019/multi_task boeck2019/multi_task_hjdb boeck2020/dar davies2009/mirex_qm_tempotracker echonest/version_3_2_1 gkiokas2012/default klapuri2006/percival2014 oliveira2010/ibt percival2014/stem scheirer1998/percival2014 schreiber2014/default schreiber2017/ismir2017 schreiber2017/mirex2017 schreiber2018/cnn schreiber2018/fcn schreiber2018/ismir2018 sun2021/default zplane/auftakt_v3
boeck2015/tempodetector2016_default 1.0000 0.0000 0.0000 0.4092 0.0000 0.0000 0.0000 0.2873 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0256 0.0000 0.5785 0.9760 0.0049
boeck2019/multi_task 0.0000 1.0000 0.0012 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0002 0.0000 0.0000 0.1614 0.0000 0.0000 0.0000 0.0000 0.0000
boeck2019/multi_task_hjdb 0.0000 0.0012 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.6815 0.0000 0.0000 0.0000 0.0000 0.0000
boeck2020/dar 0.4092 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0971 0.0016 0.0000 0.0000 0.0000 0.0000 0.0000 0.0018 0.0000 0.1298 0.3418 0.0014
davies2009/mirex_qm_tempotracker 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
echonest/version_3_2_1 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0072 0.0000 0.0000 0.6144 0.6945 0.0926 0.8350 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
gkiokas2012/default 0.0000 0.0000 0.0000 0.0000 0.0000 0.0072 1.0000 0.0000 0.0000 0.0006 0.0051 0.0000 0.0025 0.0000 0.0001 0.2040 0.0000 0.0000 0.0005
klapuri2006/percival2014 0.2873 0.0000 0.0000 0.0971 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.3244 0.0001 0.4935 0.3092 0.0241
oliveira2010/ibt 0.0000 0.0000 0.0000 0.0016 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
percival2014/stem 0.0000 0.0000 0.0000 0.0000 0.0000 0.6144 0.0006 0.0000 0.0000 1.0000 0.9737 0.1875 0.7277 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
scheirer1998/percival2014 0.0000 0.0002 0.0000 0.0000 0.0000 0.6945 0.0051 0.0000 0.0000 0.9737 1.0000 0.3727 0.8062 0.0000 0.0000 0.0001 0.0000 0.0000 0.0000
schreiber2014/default 0.0000 0.0000 0.0000 0.0000 0.0000 0.0926 0.0000 0.0000 0.0000 0.1875 0.3727 1.0000 0.0687 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2017/ismir2017 0.0000 0.0000 0.0000 0.0000 0.0000 0.8350 0.0025 0.0000 0.0000 0.7277 0.8062 0.0687 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2017/mirex2017 0.0000 0.1614 0.6815 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2018/cnn 0.0256 0.0000 0.0000 0.0018 0.0000 0.0000 0.0001 0.3244 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0007 0.0569 0.0145 0.4600
schreiber2018/fcn 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.2040 0.0001 0.0000 0.0000 0.0001 0.0000 0.0000 0.0000 0.0007 1.0000 0.0000 0.0000 0.0352
schreiber2018/ismir2018 0.5785 0.0000 0.0000 0.1298 0.0000 0.0000 0.0000 0.4935 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0569 0.0000 1.0000 0.5124 0.0113
sun2021/default 0.9760 0.0000 0.0000 0.3418 0.0000 0.0000 0.0000 0.3092 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0145 0.0000 0.5124 1.0000 0.0064
zplane/auftakt_v3 0.0049 0.0000 0.0000 0.0014 0.0000 0.0000 0.0005 0.0241 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.4600 0.0352 0.0113 0.0064 1.0000

Table 22: Paired t-test p-values, using reference annotations 1.0 as groundtruth with OE1. H0: the true mean difference between paired samples is zero. If p<=ɑ, reject H0, i.e. we have a significant difference between estimates from the two algorithms. In the table, p-values<0.05 are set in bold.

CSV JSON LATEX PICKLE

Estimator boeck2015/tempodetector2016_default boeck2019/multi_task boeck2019/multi_task_hjdb boeck2020/dar davies2009/mirex_qm_tempotracker echonest/version_3_2_1 gkiokas2012/default klapuri2006/percival2014 oliveira2010/ibt percival2014/stem scheirer1998/percival2014 schreiber2014/default schreiber2017/ismir2017 schreiber2017/mirex2017 schreiber2018/cnn schreiber2018/fcn schreiber2018/ismir2018 sun2021/default zplane/auftakt_v3
boeck2015/tempodetector2016_default 1.0000 0.7137 0.5158 0.3011 0.0000 0.1358 0.7734 0.0404 0.0008 0.4625 0.0001 0.0004 0.2434 0.4307 0.1959 0.9519 0.0278 0.0000 0.0030
boeck2019/multi_task 0.7137 1.0000 0.1404 0.5838 0.0000 0.0841 0.7017 0.0489 0.0007 0.3381 0.0001 0.0002 0.1278 0.2693 0.1001 0.7657 0.0165 0.0000 0.0022
boeck2019/multi_task_hjdb 0.5158 0.1404 1.0000 0.0998 0.0000 0.3596 0.6558 0.2205 0.0049 0.8941 0.0000 0.0030 0.4770 0.8164 0.4096 0.5202 0.0756 0.0001 0.0190
boeck2020/dar 0.3011 0.5838 0.0998 1.0000 0.0000 0.0564 0.4506 0.0109 0.0001 0.1711 0.0003 0.0000 0.0546 0.0943 0.0374 0.4347 0.0044 0.0000 0.0006
davies2009/mirex_qm_tempotracker 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.4763 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
echonest/version_3_2_1 0.1358 0.0841 0.3596 0.0564 0.0000 1.0000 0.2531 0.9281 0.1340 0.3380 0.0000 0.2323 0.6238 0.4088 0.6773 0.1297 0.6677 0.0484 0.2580
gkiokas2012/default 0.7734 0.7017 0.6558 0.4506 0.0000 0.2531 1.0000 0.1426 0.0054 0.7139 0.0001 0.0025 0.3837 0.6508 0.3739 0.7607 0.0691 0.0001 0.0185
klapuri2006/percival2014 0.0404 0.0489 0.2205 0.0109 0.0000 0.9281 0.1426 1.0000 0.0821 0.1971 0.0000 0.1878 0.4805 0.2343 0.5288 0.0376 0.6927 0.0203 0.2145
oliveira2010/ibt 0.0008 0.0007 0.0049 0.0001 0.0000 0.1340 0.0054 0.0821 1.0000 0.0056 0.0000 0.5802 0.0306 0.0070 0.0365 0.0009 0.2492 0.9155 0.7377
percival2014/stem 0.4625 0.3381 0.8941 0.1711 0.0000 0.3380 0.7139 0.1971 0.0056 1.0000 0.0000 0.0048 0.5592 0.9099 0.5325 0.4651 0.0701 0.0002 0.0191
scheirer1998/percival2014 0.0001 0.0001 0.0000 0.0003 0.4763 0.0000 0.0001 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0001 0.0000 0.0000 0.0000
schreiber2014/default 0.0004 0.0002 0.0030 0.0000 0.0000 0.2323 0.0025 0.1878 0.5802 0.0048 0.0000 1.0000 0.0118 0.0078 0.0500 0.0023 0.4452 0.4330 0.8752
schreiber2017/ismir2017 0.2434 0.1278 0.4770 0.0546 0.0000 0.6238 0.3837 0.4805 0.0306 0.5592 0.0000 0.0118 1.0000 0.5556 0.9436 0.2182 0.2793 0.0036 0.0659
schreiber2017/mirex2017 0.4307 0.2693 0.8164 0.0943 0.0000 0.4088 0.6508 0.2343 0.0070 0.9099 0.0000 0.0078 0.5556 1.0000 0.5232 0.4130 0.1588 0.0004 0.0254
schreiber2018/cnn 0.1959 0.1001 0.4096 0.0374 0.0000 0.6773 0.3739 0.5288 0.0365 0.5325 0.0000 0.0500 0.9436 0.5232 1.0000 0.1533 0.3139 0.0045 0.0827
schreiber2018/fcn 0.9519 0.7657 0.5202 0.4347 0.0000 0.1297 0.7607 0.0376 0.0009 0.4651 0.0001 0.0023 0.2182 0.4130 0.1533 1.0000 0.0224 0.0000 0.0039
schreiber2018/ismir2018 0.0278 0.0165 0.0756 0.0044 0.0000 0.6677 0.0691 0.6927 0.2492 0.0701 0.0000 0.4452 0.2793 0.1588 0.3139 0.0224 1.0000 0.1583 0.4395
sun2021/default 0.0000 0.0000 0.0001 0.0000 0.0000 0.0484 0.0001 0.0203 0.9155 0.0002 0.0000 0.4330 0.0036 0.0004 0.0045 0.0000 0.1583 1.0000 0.6372
zplane/auftakt_v3 0.0030 0.0022 0.0190 0.0006 0.0000 0.2580 0.0185 0.2145 0.7377 0.0191 0.0000 0.8752 0.0659 0.0254 0.0827 0.0039 0.4395 0.6372 1.0000

Table 23: Paired t-test p-values, using reference annotations GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI as groundtruth with OE2. H0: the true mean difference between paired samples is zero. If p<=ɑ, reject H0, i.e. we have a significant difference between estimates from the two algorithms. In the table, p-values<0.05 are set in bold.

CSV JSON LATEX PICKLE

Estimator boeck2015/tempodetector2016_default boeck2019/multi_task boeck2019/multi_task_hjdb boeck2020/dar davies2009/mirex_qm_tempotracker echonest/version_3_2_1 gkiokas2012/default klapuri2006/percival2014 oliveira2010/ibt percival2014/stem scheirer1998/percival2014 schreiber2014/default schreiber2017/ismir2017 schreiber2017/mirex2017 schreiber2018/cnn schreiber2018/fcn schreiber2018/ismir2018 sun2021/default zplane/auftakt_v3
boeck2015/tempodetector2016_default 1.0000 0.7137 0.5158 0.5876 0.0000 0.1205 0.6191 0.0873 0.0005 0.4625 0.0001 0.0002 0.2434 0.4307 0.3200 0.9519 0.0278 0.0000 0.0016
boeck2019/multi_task 0.7137 1.0000 0.1404 0.9281 0.0000 0.0821 0.5562 0.0976 0.0005 0.3381 0.0001 0.0000 0.1278 0.2693 0.1852 0.7657 0.0165 0.0000 0.0012
boeck2019/multi_task_hjdb 0.5158 0.1404 1.0000 0.2291 0.0000 0.3394 0.8035 0.3645 0.0034 0.8941 0.0000 0.0008 0.4770 0.8164 0.6238 0.5202 0.0756 0.0001 0.0119
boeck2020/dar 0.5876 0.9281 0.2291 1.0000 0.0000 0.0870 0.5294 0.0611 0.0001 0.3039 0.0001 0.0000 0.1120 0.1972 0.1543 0.7065 0.0105 0.0000 0.0006
davies2009/mirex_qm_tempotracker 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.5975 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
echonest/version_3_2_1 0.1205 0.0821 0.3394 0.0870 0.0000 1.0000 0.3087 0.8609 0.0978 0.3416 0.0000 0.1501 0.6267 0.4126 0.5082 0.1299 0.6655 0.0513 0.2125
gkiokas2012/default 0.6191 0.5562 0.8035 0.5294 0.0000 0.3087 1.0000 0.3169 0.0060 0.8652 0.0000 0.0018 0.4851 0.7878 0.6756 0.6231 0.0940 0.0002 0.0185
klapuri2006/percival2014 0.0873 0.0976 0.3645 0.0611 0.0000 0.8609 0.3169 1.0000 0.0310 0.3299 0.0000 0.0478 0.6883 0.3844 0.5259 0.0767 0.5104 0.0085 0.1014
oliveira2010/ibt 0.0005 0.0005 0.0034 0.0001 0.0000 0.0978 0.0060 0.0310 1.0000 0.0031 0.0000 0.6671 0.0223 0.0048 0.0134 0.0005 0.1871 0.9635 0.7372
percival2014/stem 0.4625 0.3381 0.8941 0.3039 0.0000 0.3416 0.8652 0.3299 0.0031 1.0000 0.0000 0.0023 0.5592 0.9099 0.7599 0.4651 0.0701 0.0002 0.0125
scheirer1998/percival2014 0.0001 0.0001 0.0000 0.0001 0.5975 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0001 0.0000 0.0000 0.0000
schreiber2014/default 0.0002 0.0000 0.0008 0.0000 0.0000 0.1501 0.0018 0.0478 0.6671 0.0023 0.0000 1.0000 0.0019 0.0016 0.0061 0.0006 0.3106 0.6360 0.9639
schreiber2017/ismir2017 0.2434 0.1278 0.4770 0.1120 0.0000 0.6267 0.4851 0.6883 0.0223 0.5592 0.0000 0.0019 1.0000 0.5556 0.7766 0.2182 0.2793 0.0036 0.0448
schreiber2017/mirex2017 0.4307 0.2693 0.8164 0.1972 0.0000 0.4126 0.7878 0.3844 0.0048 0.9099 0.0000 0.0016 0.5556 1.0000 0.8167 0.4130 0.1588 0.0004 0.0160
schreiber2018/cnn 0.3200 0.1852 0.6238 0.1543 0.0000 0.5082 0.6756 0.5259 0.0134 0.7599 0.0000 0.0061 0.7766 0.8167 1.0000 0.2672 0.1850 0.0020 0.0288
schreiber2018/fcn 0.9519 0.7657 0.5202 0.7065 0.0000 0.1299 0.6231 0.0767 0.0005 0.4651 0.0001 0.0006 0.2182 0.4130 0.2672 1.0000 0.0224 0.0000 0.0022
schreiber2018/ismir2018 0.0278 0.0165 0.0756 0.0105 0.0000 0.6655 0.0940 0.5104 0.1871 0.0701 0.0000 0.3106 0.2793 0.1588 0.1850 0.0224 1.0000 0.1583 0.3588
sun2021/default 0.0000 0.0000 0.0001 0.0000 0.0000 0.0513 0.0002 0.0085 0.9635 0.0002 0.0000 0.6360 0.0036 0.0004 0.0020 0.0000 0.1583 1.0000 0.7504
zplane/auftakt_v3 0.0016 0.0012 0.0119 0.0006 0.0000 0.2125 0.0185 0.1014 0.7372 0.0125 0.0000 0.9639 0.0448 0.0160 0.0288 0.0022 0.3588 0.7504 1.0000

Table 24: Paired t-test p-values, using reference annotations GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI as groundtruth with OE2. H0: the true mean difference between paired samples is zero. If p<=ɑ, reject H0, i.e. we have a significant difference between estimates from the two algorithms. In the table, p-values<0.05 are set in bold.

CSV JSON LATEX PICKLE

Estimator boeck2015/tempodetector2016_default boeck2019/multi_task boeck2019/multi_task_hjdb boeck2020/dar davies2009/mirex_qm_tempotracker echonest/version_3_2_1 gkiokas2012/default klapuri2006/percival2014 oliveira2010/ibt percival2014/stem scheirer1998/percival2014 schreiber2014/default schreiber2017/ismir2017 schreiber2017/mirex2017 schreiber2018/cnn schreiber2018/fcn schreiber2018/ismir2018 sun2021/default zplane/auftakt_v3
boeck2015/tempodetector2016_default 1.0000 0.6184 0.3381 0.7640 0.0000 0.0130 0.4492 0.0159 0.0020 0.7020 0.0000 0.0001 0.0441 0.2808 0.2252 0.0711 0.0710 0.0000 0.0013
boeck2019/multi_task 0.6184 1.0000 0.2720 0.7591 0.0000 0.0303 0.7649 0.0708 0.0080 0.4113 0.0000 0.0002 0.0943 0.5532 0.4308 0.1359 0.1390 0.0000 0.0080
boeck2019/multi_task_hjdb 0.3381 0.2720 1.0000 0.3964 0.0000 0.0665 0.9652 0.1261 0.0163 0.2376 0.0000 0.0004 0.1808 0.8375 0.6705 0.2826 0.2510 0.0000 0.0162
boeck2020/dar 0.7640 0.7591 0.3964 1.0000 0.0000 0.0191 0.6095 0.0271 0.0023 0.5423 0.0000 0.0001 0.0385 0.3363 0.2815 0.0859 0.0820 0.0000 0.0024
davies2009/mirex_qm_tempotracker 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.1828 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
echonest/version_3_2_1 0.0130 0.0303 0.0665 0.0191 0.0000 1.0000 0.1194 0.6542 0.5822 0.0106 0.0000 0.4033 0.4708 0.1063 0.2012 0.3398 0.4587 0.1711 0.5699
gkiokas2012/default 0.4492 0.7649 0.9652 0.6095 0.0000 0.1194 1.0000 0.1551 0.0218 0.3253 0.0000 0.0026 0.2109 0.8133 0.6842 0.3591 0.3375 0.0002 0.0228
klapuri2006/percival2014 0.0159 0.0708 0.1261 0.0271 0.0000 0.6542 0.1551 1.0000 0.2359 0.0130 0.0000 0.1241 0.7327 0.1379 0.2815 0.4989 0.7058 0.0384 0.2584
oliveira2010/ibt 0.0020 0.0080 0.0163 0.0023 0.0000 0.5822 0.0218 0.2359 1.0000 0.0011 0.0000 0.8343 0.1705 0.0155 0.0504 0.1036 0.1881 0.4865 0.9582
percival2014/stem 0.7020 0.4113 0.2376 0.5423 0.0000 0.0106 0.3253 0.0130 0.0011 1.0000 0.0000 0.0000 0.0200 0.1518 0.1800 0.0612 0.0297 0.0000 0.0021
scheirer1998/percival2014 0.0000 0.0000 0.0000 0.0000 0.1828 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2014/default 0.0001 0.0002 0.0004 0.0001 0.0000 0.4033 0.0026 0.1241 0.8343 0.0000 0.0000 1.0000 0.0125 0.0008 0.0069 0.0284 0.0617 0.5128 0.8777
schreiber2017/ismir2017 0.0441 0.0943 0.1808 0.0385 0.0000 0.4708 0.2109 0.7327 0.1705 0.0200 0.0000 0.0125 1.0000 0.1183 0.3794 0.7086 0.9329 0.0106 0.1423
schreiber2017/mirex2017 0.2808 0.5532 0.8375 0.3363 0.0000 0.1063 0.8133 0.1379 0.0155 0.1518 0.0000 0.0008 0.1183 1.0000 0.7900 0.3625 0.3338 0.0000 0.0215
schreiber2018/cnn 0.2252 0.4308 0.6705 0.2815 0.0000 0.2012 0.6842 0.2815 0.0504 0.1800 0.0000 0.0069 0.3794 0.7900 1.0000 0.5346 0.5189 0.0006 0.0373
schreiber2018/fcn 0.0711 0.1359 0.2826 0.0859 0.0000 0.3398 0.3591 0.4989 0.1036 0.0612 0.0000 0.0284 0.7086 0.3625 0.5346 1.0000 0.8162 0.0035 0.1064
schreiber2018/ismir2018 0.0710 0.1390 0.2510 0.0820 0.0000 0.4587 0.3375 0.7058 0.1881 0.0297 0.0000 0.0617 0.9329 0.3338 0.5189 0.8162 1.0000 0.0103 0.1756
sun2021/default 0.0000 0.0000 0.0000 0.0000 0.0000 0.1711 0.0002 0.0384 0.4865 0.0000 0.0000 0.5128 0.0106 0.0000 0.0006 0.0035 0.0103 1.0000 0.5201
zplane/auftakt_v3 0.0013 0.0080 0.0162 0.0024 0.0000 0.5699 0.0228 0.2584 0.9582 0.0021 0.0000 0.8777 0.1423 0.0215 0.0373 0.1064 0.1756 0.5201 1.0000

Table 25: Paired t-test p-values, using reference annotations 2.0 as groundtruth with OE2. H0: the true mean difference between paired samples is zero. If p<=ɑ, reject H0, i.e. we have a significant difference between estimates from the two algorithms. In the table, p-values<0.05 are set in bold.

CSV JSON LATEX PICKLE

Estimator boeck2015/tempodetector2016_default boeck2019/multi_task boeck2019/multi_task_hjdb boeck2020/dar davies2009/mirex_qm_tempotracker echonest/version_3_2_1 gkiokas2012/default klapuri2006/percival2014 oliveira2010/ibt percival2014/stem scheirer1998/percival2014 schreiber2014/default schreiber2017/ismir2017 schreiber2017/mirex2017 schreiber2018/cnn schreiber2018/fcn schreiber2018/ismir2018 sun2021/default zplane/auftakt_v3
boeck2015/tempodetector2016_default 1.0000 0.5201 0.2781 0.9949 0.0000 0.0102 0.1745 0.0072 0.0005 0.6177 0.0001 0.0003 0.0746 0.2797 0.0889 0.0494 0.0931 0.0000 0.0004
boeck2019/multi_task 0.5201 1.0000 0.2720 0.4276 0.0000 0.0345 0.5970 0.0681 0.0036 0.9452 0.0000 0.0008 0.1892 0.6744 0.3304 0.1359 0.2633 0.0003 0.0049
boeck2019/multi_task_hjdb 0.2781 0.2720 1.0000 0.1805 0.0000 0.0727 0.8401 0.1212 0.0076 0.6639 0.0000 0.0017 0.3299 0.9661 0.5370 0.2826 0.4269 0.0011 0.0101
boeck2020/dar 0.9949 0.4276 0.1805 1.0000 0.0000 0.0099 0.2924 0.0101 0.0003 0.5995 0.0001 0.0001 0.0385 0.2383 0.1060 0.0364 0.0890 0.0000 0.0005
davies2009/mirex_qm_tempotracker 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.5136 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
echonest/version_3_2_1 0.0102 0.0345 0.0727 0.0099 0.0000 1.0000 0.1779 0.7085 0.4875 0.0319 0.0000 0.6645 0.2639 0.0631 0.2533 0.2849 0.3126 0.4844 0.5444
gkiokas2012/default 0.1745 0.5970 0.8401 0.2924 0.0000 0.1779 1.0000 0.1974 0.0253 0.4716 0.0000 0.0284 0.6336 0.7119 0.7416 0.6388 0.6666 0.0091 0.0310
klapuri2006/percival2014 0.0072 0.0681 0.1212 0.0101 0.0000 0.7085 0.1974 1.0000 0.1883 0.0451 0.0000 0.3512 0.3934 0.0646 0.3435 0.3586 0.4595 0.2051 0.2688
oliveira2010/ibt 0.0005 0.0036 0.0076 0.0003 0.0000 0.4875 0.0253 0.1883 1.0000 0.0024 0.0000 0.7137 0.0503 0.0033 0.0478 0.0474 0.0884 0.8254 0.9616
percival2014/stem 0.6177 0.9452 0.6639 0.5995 0.0000 0.0319 0.4716 0.0451 0.0024 1.0000 0.0001 0.0011 0.1791 0.5953 0.2775 0.1931 0.1841 0.0003 0.0062
scheirer1998/percival2014 0.0001 0.0000 0.0000 0.0001 0.5136 0.0000 0.0000 0.0000 0.0000 0.0001 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2014/default 0.0003 0.0008 0.0017 0.0001 0.0000 0.6645 0.0284 0.3512 0.7137 0.0011 0.0000 1.0000 0.0125 0.0021 0.0496 0.0619 0.0911 0.8682 0.7573
schreiber2017/ismir2017 0.0746 0.1892 0.3299 0.0385 0.0000 0.2639 0.6336 0.3934 0.0503 0.1791 0.0000 0.0125 1.0000 0.2187 0.8803 0.9876 0.9613 0.0468 0.0572
schreiber2017/mirex2017 0.2797 0.6744 0.9661 0.2383 0.0000 0.0631 0.7119 0.0646 0.0033 0.5953 0.0000 0.0021 0.2187 1.0000 0.4093 0.2741 0.3830 0.0013 0.0081
schreiber2018/cnn 0.0889 0.3304 0.5370 0.1060 0.0000 0.2533 0.7416 0.3435 0.0478 0.2775 0.0000 0.0496 0.8803 0.4093 1.0000 0.8652 0.8607 0.0189 0.0508
schreiber2018/fcn 0.0494 0.1359 0.2826 0.0364 0.0000 0.2849 0.6388 0.3586 0.0474 0.1931 0.0000 0.0619 0.9876 0.2741 0.8652 1.0000 0.9519 0.0388 0.0664
schreiber2018/ismir2018 0.0931 0.2633 0.4269 0.0890 0.0000 0.3126 0.6666 0.4595 0.0884 0.1841 0.0000 0.0911 0.9613 0.3830 0.8607 0.9519 1.0000 0.0453 0.1055
sun2021/default 0.0000 0.0003 0.0011 0.0000 0.0000 0.4844 0.0091 0.2051 0.8254 0.0003 0.0000 0.8682 0.0468 0.0013 0.0189 0.0388 0.0453 1.0000 0.8724
zplane/auftakt_v3 0.0004 0.0049 0.0101 0.0005 0.0000 0.5444 0.0310 0.2688 0.9616 0.0062 0.0000 0.7573 0.0572 0.0081 0.0508 0.0664 0.1055 0.8724 1.0000

Table 26: Paired t-test p-values, using reference annotations 1.0 as groundtruth with OE2. H0: the true mean difference between paired samples is zero. If p<=ɑ, reject H0, i.e. we have a significant difference between estimates from the two algorithms. In the table, p-values<0.05 are set in bold.

CSV JSON LATEX PICKLE

OE1 on cvar-Subsets

How well does an estimator perform, when only taking tracks into account that have a cvar-value of less than τ, i.e., have a more or less stable beat?

OE1 on cvar-Subsets for 1.0 based on cvar-Values from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28

Figure 62: Mean OE1 compared to version 1.0 for tracks with cvar < τ based on beat annotations from 1.0.

CSV JSON LATEX PICKLE SVG PDF PNG

OE1 on cvar-Subsets for 2.0 based on cvar-Values from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28

Figure 63: Mean OE1 compared to version 2.0 for tracks with cvar < τ based on beat annotations from 2.0.

CSV JSON LATEX PICKLE SVG PDF PNG

OE1 on cvar-Subsets for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI based on cvar-Values from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28

Figure 64: Mean OE1 compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI for tracks with cvar < τ based on beat annotations from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI.

CSV JSON LATEX PICKLE SVG PDF PNG

OE1 on cvar-Subsets for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI based on cvar-Values from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28

Figure 65: Mean OE1 compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI for tracks with cvar < τ based on beat annotations from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI.

CSV JSON LATEX PICKLE SVG PDF PNG

OE2 on cvar-Subsets

How well does an estimator perform, when only taking tracks into account that have a cvar-value of less than τ, i.e., have a more or less stable beat?

OE2 on cvar-Subsets for 1.0 based on cvar-Values from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28

Figure 66: Mean OE2 compared to version 1.0 for tracks with cvar < τ based on beat annotations from 1.0.

CSV JSON LATEX PICKLE SVG PDF PNG

OE2 on cvar-Subsets for 2.0 based on cvar-Values from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28

Figure 67: Mean OE2 compared to version 2.0 for tracks with cvar < τ based on beat annotations from 2.0.

CSV JSON LATEX PICKLE SVG PDF PNG

OE2 on cvar-Subsets for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI based on cvar-Values from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28

Figure 68: Mean OE2 compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI for tracks with cvar < τ based on beat annotations from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI.

CSV JSON LATEX PICKLE SVG PDF PNG

OE2 on cvar-Subsets for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI based on cvar-Values from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28

Figure 69: Mean OE2 compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI for tracks with cvar < τ based on beat annotations from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI.

CSV JSON LATEX PICKLE SVG PDF PNG

OE1 on Tempo-Subsets

How well does an estimator perform, when only taking a subset of the reference annotations into account? The graphs show mean OE1 for reference subsets with tempi in [T-10,T+10] BPM. Note that the graphs do not show confidence intervals and that some values may be based on very few estimates.

OE1 on Tempo-Subsets for 1.0

Figure 70: Mean OE1 for estimates compared to version 1.0 for tempo intervals around T.

CSV JSON LATEX PICKLE SVG PDF PNG

OE1 on Tempo-Subsets for 2.0

Figure 71: Mean OE1 for estimates compared to version 2.0 for tempo intervals around T.

CSV JSON LATEX PICKLE SVG PDF PNG

OE1 on Tempo-Subsets for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI

Figure 72: Mean OE1 for estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI for tempo intervals around T.

CSV JSON LATEX PICKLE SVG PDF PNG

OE1 on Tempo-Subsets for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI

Figure 73: Mean OE1 for estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI for tempo intervals around T.

CSV JSON LATEX PICKLE SVG PDF PNG

OE2 on Tempo-Subsets

How well does an estimator perform, when only taking a subset of the reference annotations into account? The graphs show mean OE2 for reference subsets with tempi in [T-10,T+10] BPM. Note that the graphs do not show confidence intervals and that some values may be based on very few estimates.

OE2 on Tempo-Subsets for 1.0

Figure 74: Mean OE2 for estimates compared to version 1.0 for tempo intervals around T.

CSV JSON LATEX PICKLE SVG PDF PNG

OE2 on Tempo-Subsets for 2.0

Figure 75: Mean OE2 for estimates compared to version 2.0 for tempo intervals around T.

CSV JSON LATEX PICKLE SVG PDF PNG

OE2 on Tempo-Subsets for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI

Figure 76: Mean OE2 for estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI for tempo intervals around T.

CSV JSON LATEX PICKLE SVG PDF PNG

OE2 on Tempo-Subsets for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI

Figure 77: Mean OE2 for estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI for tempo intervals around T.

CSV JSON LATEX PICKLE SVG PDF PNG

Estimated OE1 for Tempo

When fitting a generalized additive model (GAM) to OE1-values and a ground truth, what OE1 can we expect with confidence?

Estimated OE1 for Tempo for 1.0

Predictions of GAMs trained on OE1 for estimates for reference 1.0.

Figure 78: OE1 predictions of a generalized additive model (GAM) fit to OE1 results for 1.0. The 95% confidence interval around the prediction is shaded in gray.

CSV JSON LATEX PICKLE SVG PDF PNG

Estimated OE1 for Tempo for 2.0

Predictions of GAMs trained on OE1 for estimates for reference 2.0.

Figure 79: OE1 predictions of a generalized additive model (GAM) fit to OE1 results for 2.0. The 95% confidence interval around the prediction is shaded in gray.

CSV JSON LATEX PICKLE SVG PDF PNG

Estimated OE1 for Tempo for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI

Predictions of GAMs trained on OE1 for estimates for reference GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI.

Figure 80: OE1 predictions of a generalized additive model (GAM) fit to OE1 results for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI. The 95% confidence interval around the prediction is shaded in gray.

CSV JSON LATEX PICKLE SVG PDF PNG

Estimated OE1 for Tempo for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI

Predictions of GAMs trained on OE1 for estimates for reference GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI.

Figure 81: OE1 predictions of a generalized additive model (GAM) fit to OE1 results for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI. The 95% confidence interval around the prediction is shaded in gray.

CSV JSON LATEX PICKLE SVG PDF PNG

Estimated OE2 for Tempo

When fitting a generalized additive model (GAM) to OE2-values and a ground truth, what OE2 can we expect with confidence?

Estimated OE2 for Tempo for 1.0

Predictions of GAMs trained on OE2 for estimates for reference 1.0.

Figure 82: OE2 predictions of a generalized additive model (GAM) fit to OE2 results for 1.0. The 95% confidence interval around the prediction is shaded in gray.

CSV JSON LATEX PICKLE SVG PDF PNG

Estimated OE2 for Tempo for 2.0

Predictions of GAMs trained on OE2 for estimates for reference 2.0.

Figure 83: OE2 predictions of a generalized additive model (GAM) fit to OE2 results for 2.0. The 95% confidence interval around the prediction is shaded in gray.

CSV JSON LATEX PICKLE SVG PDF PNG

Estimated OE2 for Tempo for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI

Predictions of GAMs trained on OE2 for estimates for reference GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI.

Figure 84: OE2 predictions of a generalized additive model (GAM) fit to OE2 results for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI. The 95% confidence interval around the prediction is shaded in gray.

CSV JSON LATEX PICKLE SVG PDF PNG

Estimated OE2 for Tempo for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI

Predictions of GAMs trained on OE2 for estimates for reference GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI.

Figure 85: OE2 predictions of a generalized additive model (GAM) fit to OE2 results for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI. The 95% confidence interval around the prediction is shaded in gray.

CSV JSON LATEX PICKLE SVG PDF PNG

OE1 for ‘tag_open’ Tags

How well does an estimator perform, when only taking tracks into account that are tagged with some kind of label? Note that some values may be based on very few estimates.

OE1 for ‘tag_open’ Tags for 1.0

Figure 86: OE1 of estimates compared to version 1.0 depending on tag from namespace ‘tag_open’.

SVG PDF PNG

OE1 for ‘tag_open’ Tags for 2.0

Figure 87: OE1 of estimates compared to version 2.0 depending on tag from namespace ‘tag_open’.

SVG PDF PNG

OE1 for ‘tag_open’ Tags for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI

Figure 88: OE1 of estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI depending on tag from namespace ‘tag_open’.

SVG PDF PNG

OE1 for ‘tag_open’ Tags for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI

Figure 89: OE1 of estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI depending on tag from namespace ‘tag_open’.

SVG PDF PNG

OE1 for ‘tag_gtzan’ Tags

How well does an estimator perform, when only taking tracks into account that are tagged with some kind of label? Note that some values may be based on very few estimates.

OE1 for ‘tag_gtzan’ Tags for 1.0

Figure 90: OE1 of estimates compared to version 1.0 depending on tag from namespace ‘tag_gtzan’.

SVG PDF PNG

OE1 for ‘tag_gtzan’ Tags for 2.0

Figure 91: OE1 of estimates compared to version 2.0 depending on tag from namespace ‘tag_gtzan’.

SVG PDF PNG

OE1 for ‘tag_gtzan’ Tags for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI

Figure 92: OE1 of estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI depending on tag from namespace ‘tag_gtzan’.

SVG PDF PNG

OE1 for ‘tag_gtzan’ Tags for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI

Figure 93: OE1 of estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI depending on tag from namespace ‘tag_gtzan’.

SVG PDF PNG

OE2 for ‘tag_open’ Tags

How well does an estimator perform, when only taking tracks into account that are tagged with some kind of label? Note that some values may be based on very few estimates.

OE2 for ‘tag_open’ Tags for 1.0

Figure 94: OE2 of estimates compared to version 1.0 depending on tag from namespace ‘tag_open’.

SVG PDF PNG

OE2 for ‘tag_open’ Tags for 2.0

Figure 95: OE2 of estimates compared to version 2.0 depending on tag from namespace ‘tag_open’.

SVG PDF PNG

OE2 for ‘tag_open’ Tags for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI

Figure 96: OE2 of estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI depending on tag from namespace ‘tag_open’.

SVG PDF PNG

OE2 for ‘tag_open’ Tags for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI

Figure 97: OE2 of estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI depending on tag from namespace ‘tag_open’.

SVG PDF PNG

OE2 for ‘tag_gtzan’ Tags

How well does an estimator perform, when only taking tracks into account that are tagged with some kind of label? Note that some values may be based on very few estimates.

OE2 for ‘tag_gtzan’ Tags for 1.0

Figure 98: OE2 of estimates compared to version 1.0 depending on tag from namespace ‘tag_gtzan’.

SVG PDF PNG

OE2 for ‘tag_gtzan’ Tags for 2.0

Figure 99: OE2 of estimates compared to version 2.0 depending on tag from namespace ‘tag_gtzan’.

SVG PDF PNG

OE2 for ‘tag_gtzan’ Tags for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI

Figure 100: OE2 of estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI depending on tag from namespace ‘tag_gtzan’.

SVG PDF PNG

OE2 for ‘tag_gtzan’ Tags for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI

Figure 101: OE2 of estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI depending on tag from namespace ‘tag_gtzan’.

SVG PDF PNG

AOE1 and AOE2

AOE1 is defined as absolute octave error between an estimate and a reference value: AOE1(E) = |log2(E/R)|.

AOE2 is the minimum of AOE1 allowing the octave errors 2, 3, 1/2, and 1/3: AOE2(E) = min(AOE1(E), AOE1(2E), AOE1(3E), AOE1(½E), AOE1(⅓E)).

Mean AOE1/AOE2 Results for 1.0

Estimator AOE1_MEAN AOE1_STDEV AOE2_MEAN AOE2_STDEV
schreiber2017/mirex2017 0.1061 0.2940 0.0185 0.0451
schreiber2017/ismir2017 0.2014 0.3816 0.0274 0.0749
boeck2019/multi_task_hjdb 0.2027 0.3837 0.0249 0.0667
percival2014/stem 0.2030 0.3847 0.0255 0.0682
schreiber2014/default 0.2055 0.3797 0.0317 0.0844
boeck2019/multi_task 0.2076 0.3849 0.0247 0.0669
schreiber2018/fcn 0.2534 0.4262 0.0238 0.0658
echonest/version_3_2_1 0.2541 0.4021 0.0461 0.1019
schreiber2018/cnn 0.2733 0.4362 0.0241 0.0702
gkiokas2012/default 0.2754 0.4439 0.0297 0.0811
zplane/auftakt_v3 0.2877 0.4449 0.0440 0.1025
klapuri2006/percival2014 0.2878 0.4435 0.0309 0.0785
schreiber2018/ismir2018 0.2986 0.4473 0.0310 0.0846
boeck2015/tempodetector2016_default 0.3026 0.4737 0.0242 0.0640
sun2021/default 0.3118 0.4431 0.0334 0.0767
boeck2020/dar 0.3126 0.4598 0.0200 0.0554
scheirer1998/percival2014 0.3355 0.4305 0.0817 0.1397
oliveira2010/ibt 0.3558 0.4684 0.0451 0.0925
davies2009/mirex_qm_tempotracker 0.4046 0.5031 0.0376 0.0693

Table 27: Mean AOE1/AOE2 for estimates compared to version 1.0 ordered by mean.

CSV JSON LATEX PICKLE

Raw data AOE1: CSV JSON LATEX PICKLE

Raw data AOE2: CSV JSON LATEX PICKLE

AOE1 distribution for 1.0

Figure 102: AOE1 for estimates compared to version 1.0. Shown are the mean AOE1 and an empirical distribution of the sample, using kernel density estimation (KDE).

CSV JSON LATEX PICKLE SVG PDF PNG

AOE2 distribution for 1.0

Figure 103: AOE2 for estimates compared to version 1.0. Shown are the mean AOE2 and an empirical distribution of the sample, using kernel density estimation (KDE).

CSV JSON LATEX PICKLE SVG PDF PNG

Mean AOE1/AOE2 Results for 2.0

Estimator AOE1_MEAN AOE1_STDEV AOE2_MEAN AOE2_STDEV
schreiber2017/mirex2017 0.1047 0.2928 0.0163 0.0404
schreiber2017/ismir2017 0.1974 0.3804 0.0252 0.0718
percival2014/stem 0.1992 0.3821 0.0247 0.0696
schreiber2014/default 0.2009 0.3776 0.0294 0.0818
boeck2019/multi_task_hjdb 0.2087 0.3903 0.0232 0.0647
boeck2019/multi_task 0.2097 0.3884 0.0230 0.0648
schreiber2018/fcn 0.2489 0.4233 0.0226 0.0664
echonest/version_3_2_1 0.2564 0.4061 0.0441 0.1004
schreiber2018/cnn 0.2695 0.4335 0.0230 0.0708
gkiokas2012/default 0.2734 0.4413 0.0273 0.0773
klapuri2006/percival2014 0.2786 0.4373 0.0295 0.0782
zplane/auftakt_v3 0.2810 0.4375 0.0427 0.1027
schreiber2018/ismir2018 0.2936 0.4457 0.0291 0.0824
boeck2015/tempodetector2016_default 0.2956 0.4695 0.0217 0.0573
sun2021/default 0.3082 0.4420 0.0317 0.0736
boeck2020/dar 0.3113 0.4607 0.0175 0.0491
scheirer1998/percival2014 0.3280 0.4262 0.0811 0.1401
oliveira2010/ibt 0.3481 0.4644 0.0443 0.0929
davies2009/mirex_qm_tempotracker 0.3940 0.4985 0.0359 0.0670

Table 28: Mean AOE1/AOE2 for estimates compared to version 2.0 ordered by mean.

CSV JSON LATEX PICKLE

Raw data AOE1: CSV JSON LATEX PICKLE

Raw data AOE2: CSV JSON LATEX PICKLE

AOE1 distribution for 2.0

Figure 104: AOE1 for estimates compared to version 2.0. Shown are the mean AOE1 and an empirical distribution of the sample, using kernel density estimation (KDE).

CSV JSON LATEX PICKLE SVG PDF PNG

AOE2 distribution for 2.0

Figure 105: AOE2 for estimates compared to version 2.0. Shown are the mean AOE2 and an empirical distribution of the sample, using kernel density estimation (KDE).

CSV JSON LATEX PICKLE SVG PDF PNG

Mean AOE1/AOE2 Results for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI

Estimator AOE1_MEAN AOE1_STDEV AOE2_MEAN AOE2_STDEV
boeck2020/dar 0.1383 0.3387 0.0168 0.0616
sun2021/default 0.1715 0.3573 0.0302 0.0769
schreiber2018/ismir2018 0.2012 0.3844 0.0291 0.0873
schreiber2018/cnn 0.2163 0.4023 0.0229 0.0733
boeck2015/tempodetector2016_default 0.2167 0.4143 0.0209 0.0649
schreiber2018/fcn 0.2320 0.4139 0.0206 0.0647
klapuri2006/percival2014 0.2683 0.4353 0.0233 0.0785
oliveira2010/ibt 0.2702 0.4239 0.0410 0.0898
schreiber2017/ismir2017 0.2711 0.4371 0.0248 0.0811
boeck2019/multi_task 0.2849 0.4470 0.0212 0.0708
davies2009/mirex_qm_tempotracker 0.2851 0.4443 0.0365 0.0680
zplane/auftakt_v3 0.2867 0.4407 0.0393 0.1031
schreiber2014/default 0.2896 0.4405 0.0250 0.0806
echonest/version_3_2_1 0.2920 0.4405 0.0415 0.1030
percival2014/stem 0.2988 0.4546 0.0207 0.0703
boeck2019/multi_task_hjdb 0.3119 0.4639 0.0218 0.0719
schreiber2017/mirex2017 0.3143 0.4677 0.0184 0.0671
gkiokas2012/default 0.3276 0.4723 0.0261 0.0778
scheirer1998/percival2014 0.3983 0.4684 0.0770 0.1380

Table 29: Mean AOE1/AOE2 for estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI ordered by mean.

CSV JSON LATEX PICKLE

Raw data AOE1: CSV JSON LATEX PICKLE

Raw data AOE2: CSV JSON LATEX PICKLE

AOE1 distribution for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI

Figure 106: AOE1 for estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI. Shown are the mean AOE1 and an empirical distribution of the sample, using kernel density estimation (KDE).

CSV JSON LATEX PICKLE SVG PDF PNG

AOE2 distribution for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI

Figure 107: AOE2 for estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI. Shown are the mean AOE2 and an empirical distribution of the sample, using kernel density estimation (KDE).

CSV JSON LATEX PICKLE SVG PDF PNG

Mean AOE1/AOE2 Results for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI

Estimator AOE1_MEAN AOE1_STDEV AOE2_MEAN AOE2_STDEV
boeck2020/dar 0.1367 0.3390 0.0155 0.0632
sun2021/default 0.1708 0.3575 0.0294 0.0765
schreiber2018/ismir2018 0.2002 0.3848 0.0280 0.0882
schreiber2018/cnn 0.2154 0.4030 0.0220 0.0736
boeck2015/tempodetector2016_default 0.2164 0.4148 0.0206 0.0656
schreiber2018/fcn 0.2309 0.4143 0.0197 0.0658
schreiber2017/ismir2017 0.2696 0.4378 0.0231 0.0820
oliveira2010/ibt 0.2697 0.4244 0.0400 0.0892
klapuri2006/percival2014 0.2699 0.4345 0.0247 0.0781
boeck2019/multi_task 0.2836 0.4470 0.0202 0.0720
zplane/auftakt_v3 0.2854 0.4413 0.0378 0.1036
davies2009/mirex_qm_tempotracker 0.2855 0.4445 0.0368 0.0673
schreiber2014/default 0.2877 0.4413 0.0227 0.0806
echonest/version_3_2_1 0.2908 0.4410 0.0403 0.1038
percival2014/stem 0.2976 0.4550 0.0190 0.0708
boeck2019/multi_task_hjdb 0.3108 0.4639 0.0211 0.0733
schreiber2017/mirex2017 0.3126 0.4684 0.0166 0.0681
gkiokas2012/default 0.3268 0.4733 0.0250 0.0783
scheirer1998/percival2014 0.3981 0.4686 0.0767 0.1384

Table 30: Mean AOE1/AOE2 for estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI ordered by mean.

CSV JSON LATEX PICKLE

Raw data AOE1: CSV JSON LATEX PICKLE

Raw data AOE2: CSV JSON LATEX PICKLE

AOE1 distribution for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI

Figure 108: AOE1 for estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI. Shown are the mean AOE1 and an empirical distribution of the sample, using kernel density estimation (KDE).

CSV JSON LATEX PICKLE SVG PDF PNG

AOE2 distribution for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI

Figure 109: AOE2 for estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI. Shown are the mean AOE2 and an empirical distribution of the sample, using kernel density estimation (KDE).

CSV JSON LATEX PICKLE SVG PDF PNG

Significance of Differences

Estimator boeck2015/tempodetector2016_default boeck2019/multi_task boeck2019/multi_task_hjdb boeck2020/dar davies2009/mirex_qm_tempotracker echonest/version_3_2_1 gkiokas2012/default klapuri2006/percival2014 oliveira2010/ibt percival2014/stem scheirer1998/percival2014 schreiber2014/default schreiber2017/ismir2017 schreiber2017/mirex2017 schreiber2018/cnn schreiber2018/fcn schreiber2018/ismir2018 sun2021/default zplane/auftakt_v3
boeck2015/tempodetector2016_default 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0001 0.0001 0.0000 0.0000 0.0000 0.0001 0.0000 0.9391 0.2842 0.1722 0.0002 0.0000
boeck2019/multi_task 0.0000 1.0000 0.0002 0.0000 0.9005 0.5942 0.0047 0.3470 0.3703 0.2949 0.0000 0.7233 0.2313 0.0249 0.0000 0.0002 0.0000 0.0000 0.9020
boeck2019/multi_task_hjdb 0.0000 0.0002 1.0000 0.0000 0.1625 0.1348 0.3083 0.0076 0.0133 0.3092 0.0000 0.0552 0.0010 0.8955 0.0000 0.0000 0.0000 0.0000 0.0857
boeck2020/dar 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0015 0.0000
davies2009/mirex_qm_tempotracker 0.0000 0.9005 0.1625 0.0000 1.0000 0.7506 0.0102 0.1564 0.1033 0.4191 0.0000 0.8895 0.2963 0.1219 0.0000 0.0004 0.0000 0.0000 0.9922
echonest/version_3_2_1 0.0000 0.5942 0.1348 0.0000 0.7506 1.0000 0.0130 0.1110 0.1297 0.6375 0.0000 0.7522 0.0823 0.1170 0.0000 0.0000 0.0000 0.0000 0.6533
gkiokas2012/default 0.0000 0.0047 0.3083 0.0000 0.0102 0.0130 1.0000 0.0001 0.0002 0.0305 0.0000 0.0032 0.0000 0.3301 0.0000 0.0000 0.0000 0.0000 0.0034
klapuri2006/percival2014 0.0001 0.3470 0.0076 0.0000 0.1564 0.1110 0.0001 1.0000 0.9828 0.0107 0.0000 0.1359 0.9854 0.0029 0.0001 0.0047 0.0000 0.0000 0.1374
oliveira2010/ibt 0.0001 0.3703 0.0133 0.0000 0.1033 0.1297 0.0002 0.9828 1.0000 0.0380 0.0000 0.1984 0.9986 0.0082 0.0001 0.0082 0.0000 0.0000 0.1408
percival2014/stem 0.0000 0.2949 0.3092 0.0000 0.4191 0.6375 0.0305 0.0107 0.0380 1.0000 0.0000 0.3438 0.0141 0.2379 0.0000 0.0000 0.0000 0.0000 0.2803
scheirer1998/percival2014 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2014/default 0.0000 0.7233 0.0552 0.0000 0.8895 0.7522 0.0032 0.1359 0.1984 0.3438 0.0000 1.0000 0.0636 0.0382 0.0000 0.0000 0.0000 0.0000 0.8514
schreiber2017/ismir2017 0.0001 0.2313 0.0010 0.0000 0.2963 0.0823 0.0000 0.9854 0.9986 0.0141 0.0000 0.0636 1.0000 0.0001 0.0000 0.0026 0.0000 0.0000 0.1848
schreiber2017/mirex2017 0.0000 0.0249 0.8955 0.0000 0.1219 0.1170 0.3301 0.0029 0.0082 0.2379 0.0000 0.0382 0.0001 1.0000 0.0000 0.0000 0.0000 0.0000 0.0559
schreiber2018/cnn 0.9391 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0001 0.0001 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.1895 0.1780 0.0001 0.0000
schreiber2018/fcn 0.2842 0.0002 0.0000 0.0000 0.0004 0.0000 0.0000 0.0047 0.0082 0.0000 0.0000 0.0000 0.0026 0.0000 0.1895 1.0000 0.0112 0.0000 0.0001
schreiber2018/ismir2018 0.1722 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.1780 0.0112 1.0000 0.0075 0.0000
sun2021/default 0.0002 0.0000 0.0000 0.0015 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0001 0.0000 0.0075 1.0000 0.0000
zplane/auftakt_v3 0.0000 0.9020 0.0857 0.0000 0.9922 0.6533 0.0034 0.1374 0.1408 0.2803 0.0000 0.8514 0.1848 0.0559 0.0000 0.0001 0.0000 0.0000 1.0000

Table 31: Paired t-test p-values, using reference annotations GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI as groundtruth with AOE1. H0: the true mean difference between paired samples is zero. If p<=ɑ, reject H0, i.e. we have a significant difference between estimates from the two algorithms. In the table, p-values<0.05 are set in bold.

CSV JSON LATEX PICKLE

Estimator boeck2015/tempodetector2016_default boeck2019/multi_task boeck2019/multi_task_hjdb boeck2020/dar davies2009/mirex_qm_tempotracker echonest/version_3_2_1 gkiokas2012/default klapuri2006/percival2014 oliveira2010/ibt percival2014/stem scheirer1998/percival2014 schreiber2014/default schreiber2017/ismir2017 schreiber2017/mirex2017 schreiber2018/cnn schreiber2018/fcn schreiber2018/ismir2018 sun2021/default zplane/auftakt_v3
boeck2015/tempodetector2016_default 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0002 0.0001 0.0000 0.0000 0.0000 0.0000 0.0000 0.9725 0.2597 0.1900 0.0002 0.0000
boeck2019/multi_task 0.0000 1.0000 0.0002 0.0000 0.9814 0.5990 0.0050 0.2563 0.3442 0.2999 0.0000 0.6847 0.2370 0.0229 0.0000 0.0002 0.0000 0.0000 0.9018
boeck2019/multi_task_hjdb 0.0000 0.0002 1.0000 0.0000 0.1380 0.1368 0.3149 0.0045 0.0120 0.3120 0.0000 0.0636 0.0011 0.8589 0.0000 0.0000 0.0000 0.0000 0.0880
boeck2020/dar 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0019 0.0000
davies2009/mirex_qm_tempotracker 0.0000 0.9814 0.1380 0.0000 1.0000 0.6708 0.0080 0.1306 0.1275 0.3598 0.0000 0.7730 0.3598 0.0946 0.0000 0.0005 0.0000 0.0000 0.9004
echonest/version_3_2_1 0.0000 0.5990 0.1368 0.0000 0.6708 1.0000 0.0138 0.0727 0.1179 0.6396 0.0000 0.7952 0.0859 0.1079 0.0000 0.0000 0.0000 0.0000 0.6583
gkiokas2012/default 0.0000 0.0050 0.3149 0.0000 0.0080 0.0138 1.0000 0.0000 0.0002 0.0322 0.0000 0.0039 0.0000 0.3591 0.0000 0.0000 0.0000 0.0000 0.0036
klapuri2006/percival2014 0.0002 0.2563 0.0045 0.0000 0.1306 0.0727 0.0000 1.0000 0.8308 0.0051 0.0000 0.0750 0.8093 0.0013 0.0001 0.0084 0.0000 0.0000 0.0787
oliveira2010/ibt 0.0001 0.3442 0.0120 0.0000 0.1275 0.1179 0.0002 0.8308 1.0000 0.0337 0.0000 0.1658 0.9453 0.0065 0.0001 0.0091 0.0000 0.0000 0.1224
percival2014/stem 0.0000 0.2999 0.3120 0.0000 0.3598 0.6396 0.0322 0.0051 0.0337 1.0000 0.0000 0.3791 0.0152 0.2207 0.0000 0.0000 0.0000 0.0000 0.2848
scheirer1998/percival2014 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2014/default 0.0000 0.6847 0.0636 0.0000 0.7730 0.7952 0.0039 0.0750 0.1658 0.3791 0.0000 1.0000 0.0572 0.0392 0.0000 0.0000 0.0000 0.0000 0.8139
schreiber2017/ismir2017 0.0000 0.2370 0.0011 0.0000 0.3598 0.0859 0.0000 0.8093 0.9453 0.0152 0.0000 0.0572 1.0000 0.0001 0.0000 0.0023 0.0000 0.0000 0.1892
schreiber2017/mirex2017 0.0000 0.0229 0.8589 0.0000 0.0946 0.1079 0.3591 0.0013 0.0065 0.2207 0.0000 0.0392 0.0001 1.0000 0.0000 0.0000 0.0000 0.0000 0.0519
schreiber2018/cnn 0.9725 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0001 0.0001 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.1830 0.1824 0.0001 0.0000
schreiber2018/fcn 0.2597 0.0002 0.0000 0.0000 0.0005 0.0000 0.0000 0.0084 0.0091 0.0000 0.0000 0.0000 0.0023 0.0000 0.1830 1.0000 0.0110 0.0000 0.0001
schreiber2018/ismir2018 0.1900 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.1824 0.0110 1.0000 0.0068 0.0000
sun2021/default 0.0002 0.0000 0.0000 0.0019 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0001 0.0000 0.0068 1.0000 0.0000
zplane/auftakt_v3 0.0000 0.9018 0.0880 0.0000 0.9004 0.6583 0.0036 0.0787 0.1224 0.2848 0.0000 0.8139 0.1892 0.0519 0.0000 0.0001 0.0000 0.0000 1.0000

Table 32: Paired t-test p-values, using reference annotations GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI as groundtruth with AOE1. H0: the true mean difference between paired samples is zero. If p<=ɑ, reject H0, i.e. we have a significant difference between estimates from the two algorithms. In the table, p-values<0.05 are set in bold.

CSV JSON LATEX PICKLE

Estimator boeck2015/tempodetector2016_default boeck2019/multi_task boeck2019/multi_task_hjdb boeck2020/dar davies2009/mirex_qm_tempotracker echonest/version_3_2_1 gkiokas2012/default klapuri2006/percival2014 oliveira2010/ibt percival2014/stem scheirer1998/percival2014 schreiber2014/default schreiber2017/ismir2017 schreiber2017/mirex2017 schreiber2018/cnn schreiber2018/fcn schreiber2018/ismir2018 sun2021/default zplane/auftakt_v3
boeck2015/tempodetector2016_default 1.0000 0.0000 0.0000 0.2078 0.0000 0.0079 0.1432 0.2123 0.0001 0.0000 0.0434 0.0000 0.0000 0.0000 0.0362 0.0005 0.8661 0.3020 0.2708
boeck2019/multi_task 0.0000 1.0000 0.8940 0.0000 0.0000 0.0003 0.0000 0.0000 0.0000 0.4193 0.0000 0.4388 0.3052 0.0000 0.0000 0.0053 0.0000 0.0000 0.0000
boeck2019/multi_task_hjdb 0.0000 0.8940 1.0000 0.0000 0.0000 0.0003 0.0000 0.0000 0.0000 0.4710 0.0000 0.5132 0.3703 0.0000 0.0001 0.0064 0.0000 0.0000 0.0000
boeck2020/dar 0.2078 0.0000 0.0000 1.0000 0.0000 0.0002 0.0206 0.0260 0.0115 0.0000 0.2741 0.0000 0.0000 0.0000 0.0007 0.0000 0.1219 0.7688 0.0431
davies2009/mirex_qm_tempotracker 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0001 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
echonest/version_3_2_1 0.0079 0.0003 0.0003 0.0002 0.0000 1.0000 0.2191 0.0959 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.3847 0.5540 0.0078 0.0002 0.0695
gkiokas2012/default 0.1432 0.0000 0.0000 0.0206 0.0000 0.2191 1.0000 0.7094 0.0000 0.0000 0.0003 0.0000 0.0000 0.0000 0.7905 0.1013 0.1783 0.0218 0.5884
klapuri2006/percival2014 0.2123 0.0000 0.0000 0.0260 0.0000 0.0959 0.7094 1.0000 0.0000 0.0000 0.0004 0.0000 0.0000 0.0000 0.5017 0.0309 0.1960 0.0337 0.8201
oliveira2010/ibt 0.0001 0.0000 0.0000 0.0115 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.2211 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0033 0.0000
percival2014/stem 0.0000 0.4193 0.4710 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.8716 0.8769 0.0000 0.0000 0.0002 0.0000 0.0000 0.0000
scheirer1998/percival2014 0.0434 0.0000 0.0000 0.2741 0.0001 0.0000 0.0003 0.0004 0.2211 0.0000 1.0000 0.0000 0.0000 0.0000 0.0002 0.0000 0.0181 0.1872 0.0009
schreiber2014/default 0.0000 0.4388 0.5132 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.8716 0.0000 1.0000 0.7199 0.0000 0.0000 0.0002 0.0000 0.0000 0.0000
schreiber2017/ismir2017 0.0000 0.3052 0.3703 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.8769 0.0000 0.7199 1.0000 0.0000 0.0000 0.0001 0.0000 0.0000 0.0000
schreiber2017/mirex2017 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2018/cnn 0.0362 0.0000 0.0001 0.0007 0.0000 0.3847 0.7905 0.5017 0.0000 0.0000 0.0002 0.0000 0.0000 0.0000 1.0000 0.0819 0.0326 0.0009 0.3906
schreiber2018/fcn 0.0005 0.0053 0.0064 0.0000 0.0000 0.5540 0.1013 0.0309 0.0000 0.0002 0.0000 0.0002 0.0001 0.0000 0.0819 1.0000 0.0002 0.0000 0.0224
schreiber2018/ismir2018 0.8661 0.0000 0.0000 0.1219 0.0000 0.0078 0.1783 0.1960 0.0000 0.0000 0.0181 0.0000 0.0000 0.0000 0.0326 0.0002 1.0000 0.1792 0.3066
sun2021/default 0.3020 0.0000 0.0000 0.7688 0.0000 0.0002 0.0218 0.0337 0.0033 0.0000 0.1872 0.0000 0.0000 0.0000 0.0009 0.0000 0.1792 1.0000 0.0479
zplane/auftakt_v3 0.2708 0.0000 0.0000 0.0431 0.0000 0.0695 0.5884 0.8201 0.0000 0.0000 0.0009 0.0000 0.0000 0.0000 0.3906 0.0224 0.3066 0.0479 1.0000

Table 33: Paired t-test p-values, using reference annotations 2.0 as groundtruth with AOE1. H0: the true mean difference between paired samples is zero. If p<=ɑ, reject H0, i.e. we have a significant difference between estimates from the two algorithms. In the table, p-values<0.05 are set in bold.

CSV JSON LATEX PICKLE

Estimator boeck2015/tempodetector2016_default boeck2019/multi_task boeck2019/multi_task_hjdb boeck2020/dar davies2009/mirex_qm_tempotracker echonest/version_3_2_1 gkiokas2012/default klapuri2006/percival2014 oliveira2010/ibt percival2014/stem scheirer1998/percival2014 schreiber2014/default schreiber2017/ismir2017 schreiber2017/mirex2017 schreiber2018/cnn schreiber2018/fcn schreiber2018/ismir2018 sun2021/default zplane/auftakt_v3
boeck2015/tempodetector2016_default 1.0000 0.0000 0.0000 0.3794 0.0000 0.0010 0.0712 0.2748 0.0001 0.0000 0.0407 0.0000 0.0000 0.0000 0.0184 0.0002 0.7333 0.4035 0.2602
boeck2019/multi_task 0.0000 1.0000 0.4991 0.0000 0.0000 0.0003 0.0000 0.0000 0.0000 0.7256 0.0000 0.7856 0.6012 0.0000 0.0000 0.0014 0.0000 0.0000 0.0000
boeck2019/multi_task_hjdb 0.0000 0.4991 1.0000 0.0000 0.0000 0.0001 0.0000 0.0000 0.0000 0.9808 0.0000 0.8806 0.9162 0.0000 0.0000 0.0007 0.0000 0.0000 0.0000
boeck2020/dar 0.3794 0.0000 0.0000 1.0000 0.0000 0.0001 0.0219 0.0858 0.0035 0.0000 0.1635 0.0000 0.0000 0.0000 0.0011 0.0000 0.1861 0.9391 0.0842
davies2009/mirex_qm_tempotracker 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
echonest/version_3_2_1 0.0010 0.0003 0.0001 0.0001 0.0000 1.0000 0.1246 0.0121 0.0000 0.0001 0.0000 0.0000 0.0000 0.0000 0.1910 0.9167 0.0014 0.0000 0.0145
gkiokas2012/default 0.0712 0.0000 0.0000 0.0219 0.0000 0.1246 1.0000 0.3734 0.0000 0.0000 0.0001 0.0000 0.0000 0.0000 0.8870 0.1392 0.1207 0.0146 0.3790
klapuri2006/percival2014 0.2748 0.0000 0.0000 0.0858 0.0000 0.0121 0.3734 1.0000 0.0000 0.0000 0.0007 0.0000 0.0000 0.0000 0.2874 0.0123 0.3514 0.0806 0.9923
oliveira2010/ibt 0.0001 0.0000 0.0000 0.0035 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.2105 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0015 0.0000
percival2014/stem 0.0000 0.7256 0.9808 0.0000 0.0000 0.0001 0.0000 0.0000 0.0000 1.0000 0.0000 0.8121 0.8837 0.0000 0.0000 0.0002 0.0000 0.0000 0.0000
scheirer1998/percival2014 0.0407 0.0000 0.0000 0.1635 0.0000 0.0000 0.0001 0.0007 0.2105 0.0000 1.0000 0.0000 0.0000 0.0000 0.0001 0.0000 0.0116 0.1360 0.0008
schreiber2014/default 0.0000 0.7856 0.8806 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.8121 0.0000 1.0000 0.6671 0.0000 0.0000 0.0001 0.0000 0.0000 0.0000
schreiber2017/ismir2017 0.0000 0.6012 0.9162 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.8837 0.0000 0.6671 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2017/mirex2017 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2018/cnn 0.0184 0.0000 0.0000 0.0011 0.0000 0.1910 0.8870 0.2874 0.0000 0.0000 0.0001 0.0000 0.0000 0.0000 1.0000 0.0916 0.0246 0.0006 0.2835
schreiber2018/fcn 0.0002 0.0014 0.0007 0.0000 0.0000 0.9167 0.1392 0.0123 0.0000 0.0002 0.0000 0.0001 0.0000 0.0000 0.0916 1.0000 0.0002 0.0000 0.0146
schreiber2018/ismir2018 0.7333 0.0000 0.0000 0.1861 0.0000 0.0014 0.1207 0.3514 0.0000 0.0000 0.0116 0.0000 0.0000 0.0000 0.0246 0.0002 1.0000 0.1855 0.3778
sun2021/default 0.4035 0.0000 0.0000 0.9391 0.0000 0.0000 0.0146 0.0806 0.0015 0.0000 0.1360 0.0000 0.0000 0.0000 0.0006 0.0000 0.1855 1.0000 0.0708
zplane/auftakt_v3 0.2602 0.0000 0.0000 0.0842 0.0000 0.0145 0.3790 0.9923 0.0000 0.0000 0.0008 0.0000 0.0000 0.0000 0.2835 0.0146 0.3778 0.0708 1.0000

Table 34: Paired t-test p-values, using reference annotations 1.0 as groundtruth with AOE1. H0: the true mean difference between paired samples is zero. If p<=ɑ, reject H0, i.e. we have a significant difference between estimates from the two algorithms. In the table, p-values<0.05 are set in bold.

CSV JSON LATEX PICKLE

Estimator boeck2015/tempodetector2016_default boeck2019/multi_task boeck2019/multi_task_hjdb boeck2020/dar davies2009/mirex_qm_tempotracker echonest/version_3_2_1 gkiokas2012/default klapuri2006/percival2014 oliveira2010/ibt percival2014/stem scheirer1998/percival2014 schreiber2014/default schreiber2017/ismir2017 schreiber2017/mirex2017 schreiber2018/cnn schreiber2018/fcn schreiber2018/ismir2018 sun2021/default zplane/auftakt_v3
boeck2015/tempodetector2016_default 1.0000 0.9802 0.6937 0.0055 0.0000 0.0000 0.0609 0.1140 0.0000 0.4637 0.0000 0.4024 0.3194 0.0787 0.5312 0.6648 0.0029 0.0000 0.0000
boeck2019/multi_task 0.9802 1.0000 0.2592 0.0046 0.0000 0.0000 0.0713 0.1099 0.0000 0.4914 0.0000 0.3660 0.3016 0.0992 0.5797 0.6967 0.0071 0.0000 0.0000
boeck2019/multi_task_hjdb 0.6937 0.2592 1.0000 0.0012 0.0000 0.0000 0.1400 0.2160 0.0000 0.2963 0.0000 0.5859 0.4816 0.0385 0.8438 0.4281 0.0171 0.0002 0.0000
boeck2020/dar 0.0055 0.0046 0.0012 1.0000 0.0000 0.0000 0.0002 0.0006 0.0000 0.1561 0.0000 0.0073 0.0038 0.7214 0.0099 0.0619 0.0000 0.0000 0.0000
davies2009/mirex_qm_tempotracker 0.0000 0.0000 0.0000 0.0000 1.0000 0.2834 0.0000 0.0000 0.2863 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0022 0.0049 0.7531
echonest/version_3_2_1 0.0000 0.0000 0.0000 0.0000 0.2834 1.0000 0.0000 0.0000 0.9451 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0002 0.0003 0.4972
gkiokas2012/default 0.0609 0.0713 0.1400 0.0002 0.0000 0.0000 1.0000 0.9165 0.0000 0.0199 0.0000 0.3939 0.5209 0.0014 0.2768 0.0408 0.2785 0.0789 0.0002
klapuri2006/percival2014 0.1140 0.1099 0.2160 0.0006 0.0000 0.0000 0.9165 1.0000 0.0000 0.0302 0.0000 0.4501 0.5844 0.0029 0.3249 0.0392 0.2167 0.0675 0.0000
oliveira2010/ibt 0.0000 0.0000 0.0000 0.0000 0.2863 0.9451 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0002 0.0008 0.4984
percival2014/stem 0.4637 0.4914 0.2963 0.1561 0.0000 0.0000 0.0199 0.0302 0.0000 1.0000 0.0000 0.1285 0.1039 0.3113 0.1855 0.7800 0.0004 0.0000 0.0000
scheirer1998/percival2014 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2014/default 0.4024 0.3660 0.5859 0.0073 0.0000 0.0000 0.3939 0.4501 0.0000 0.1285 0.0000 1.0000 0.8192 0.0108 0.7703 0.2426 0.0489 0.0029 0.0000
schreiber2017/ismir2017 0.3194 0.3016 0.4816 0.0038 0.0000 0.0000 0.5209 0.5844 0.0000 0.1039 0.0000 0.8192 1.0000 0.0018 0.6216 0.1590 0.0611 0.0092 0.0000
schreiber2017/mirex2017 0.0787 0.0992 0.0385 0.7214 0.0000 0.0000 0.0014 0.0029 0.0000 0.3113 0.0000 0.0108 0.0018 1.0000 0.0174 0.1830 0.0000 0.0000 0.0000
schreiber2018/cnn 0.5312 0.5797 0.8438 0.0099 0.0000 0.0000 0.2768 0.3249 0.0000 0.1855 0.0000 0.7703 0.6216 0.0174 1.0000 0.2625 0.0052 0.0017 0.0000
schreiber2018/fcn 0.6648 0.6967 0.4281 0.0619 0.0000 0.0000 0.0408 0.0392 0.0000 0.7800 0.0000 0.2426 0.1590 0.1830 0.2625 1.0000 0.0008 0.0000 0.0000
schreiber2018/ismir2018 0.0029 0.0071 0.0171 0.0000 0.0022 0.0002 0.2785 0.2167 0.0002 0.0004 0.0000 0.0489 0.0611 0.0000 0.0052 0.0008 1.0000 0.4726 0.0036
sun2021/default 0.0000 0.0000 0.0002 0.0000 0.0049 0.0003 0.0789 0.0675 0.0008 0.0000 0.0000 0.0029 0.0092 0.0000 0.0017 0.0000 0.4726 1.0000 0.0086
zplane/auftakt_v3 0.0000 0.0000 0.0000 0.0000 0.7531 0.4972 0.0002 0.0000 0.4984 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0036 0.0086 1.0000

Table 35: Paired t-test p-values, using reference annotations GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI as groundtruth with AOE2. H0: the true mean difference between paired samples is zero. If p<=ɑ, reject H0, i.e. we have a significant difference between estimates from the two algorithms. In the table, p-values<0.05 are set in bold.

CSV JSON LATEX PICKLE

Estimator boeck2015/tempodetector2016_default boeck2019/multi_task boeck2019/multi_task_hjdb boeck2020/dar davies2009/mirex_qm_tempotracker echonest/version_3_2_1 gkiokas2012/default klapuri2006/percival2014 oliveira2010/ibt percival2014/stem scheirer1998/percival2014 schreiber2014/default schreiber2017/ismir2017 schreiber2017/mirex2017 schreiber2018/cnn schreiber2018/fcn schreiber2018/ismir2018 sun2021/default zplane/auftakt_v3
boeck2015/tempodetector2016_default 1.0000 0.7648 0.5676 0.0275 0.0000 0.0000 0.0275 0.3498 0.0000 0.9268 0.0000 0.0972 0.1262 0.2603 0.3670 0.8952 0.0012 0.0000 0.0000
boeck2019/multi_task 0.7648 1.0000 0.4679 0.0082 0.0000 0.0000 0.0639 0.4758 0.0000 0.7131 0.0000 0.1400 0.1914 0.1787 0.5964 0.6874 0.0066 0.0000 0.0000
boeck2019/multi_task_hjdb 0.5676 0.4679 1.0000 0.0043 0.0000 0.0000 0.0986 0.6238 0.0000 0.5488 0.0000 0.2102 0.2665 0.1048 0.7634 0.5073 0.0116 0.0002 0.0000
boeck2020/dar 0.0275 0.0082 0.0043 1.0000 0.0000 0.0000 0.0003 0.0159 0.0000 0.1076 0.0000 0.0019 0.0023 0.5872 0.0160 0.0881 0.0000 0.0000 0.0000
davies2009/mirex_qm_tempotracker 0.0000 0.0000 0.0000 0.0000 1.0000 0.1251 0.0001 0.0000 0.1408 0.0000 0.0000 0.0001 0.0001 0.0000 0.0000 0.0000 0.0109 0.0199 0.3996
echonest/version_3_2_1 0.0000 0.0000 0.0000 0.0000 0.1251 1.0000 0.0000 0.0000 0.8957 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0001 0.0001 0.5477
gkiokas2012/default 0.0275 0.0639 0.0986 0.0003 0.0001 0.0000 1.0000 0.3337 0.0000 0.0357 0.0000 0.6938 0.6567 0.0031 0.2531 0.0348 0.2910 0.1071 0.0001
klapuri2006/percival2014 0.3498 0.4758 0.6238 0.0159 0.0000 0.0000 0.3337 1.0000 0.0000 0.3200 0.0000 0.5066 0.5885 0.0664 0.8823 0.2670 0.0364 0.0094 0.0000
oliveira2010/ibt 0.0000 0.0000 0.0000 0.0000 0.1408 0.8957 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0002 0.0007 0.6048
percival2014/stem 0.9268 0.7131 0.5488 0.1076 0.0000 0.0000 0.0357 0.3200 0.0000 1.0000 0.0000 0.0720 0.1033 0.3133 0.3289 0.9698 0.0010 0.0000 0.0000
scheirer1998/percival2014 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2014/default 0.0972 0.1400 0.2102 0.0019 0.0001 0.0000 0.6938 0.5066 0.0000 0.0720 0.0000 1.0000 0.9126 0.0051 0.3887 0.0868 0.1320 0.0213 0.0000
schreiber2017/ismir2017 0.1262 0.1914 0.2665 0.0023 0.0001 0.0000 0.6567 0.5885 0.0000 0.1033 0.0000 0.9126 1.0000 0.0018 0.4191 0.0888 0.1012 0.0235 0.0000
schreiber2017/mirex2017 0.2603 0.1787 0.1048 0.5872 0.0000 0.0000 0.0031 0.0664 0.0000 0.3133 0.0000 0.0051 0.0018 1.0000 0.0433 0.3166 0.0000 0.0000 0.0000
schreiber2018/cnn 0.3670 0.5964 0.7634 0.0160 0.0000 0.0000 0.2531 0.8823 0.0000 0.3289 0.0000 0.3887 0.4191 0.0433 1.0000 0.2688 0.0045 0.0022 0.0000
schreiber2018/fcn 0.8952 0.6874 0.5073 0.0881 0.0000 0.0000 0.0348 0.2670 0.0000 0.9698 0.0000 0.0868 0.0888 0.3166 0.2688 1.0000 0.0007 0.0000 0.0000
schreiber2018/ismir2018 0.0012 0.0066 0.0116 0.0000 0.0109 0.0001 0.2910 0.0364 0.0002 0.0010 0.0000 0.1320 0.1012 0.0000 0.0045 0.0007 1.0000 0.5398 0.0023
sun2021/default 0.0000 0.0000 0.0002 0.0000 0.0199 0.0001 0.1071 0.0094 0.0007 0.0000 0.0000 0.0213 0.0235 0.0000 0.0022 0.0000 0.5398 1.0000 0.0044
zplane/auftakt_v3 0.0000 0.0000 0.0000 0.0000 0.3996 0.5477 0.0001 0.0000 0.6048 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0023 0.0044 1.0000

Table 36: Paired t-test p-values, using reference annotations GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI as groundtruth with AOE2. H0: the true mean difference between paired samples is zero. If p<=ɑ, reject H0, i.e. we have a significant difference between estimates from the two algorithms. In the table, p-values<0.05 are set in bold.

CSV JSON LATEX PICKLE

Estimator boeck2015/tempodetector2016_default boeck2019/multi_task boeck2019/multi_task_hjdb boeck2020/dar davies2009/mirex_qm_tempotracker echonest/version_3_2_1 gkiokas2012/default klapuri2006/percival2014 oliveira2010/ibt percival2014/stem scheirer1998/percival2014 schreiber2014/default schreiber2017/ismir2017 schreiber2017/mirex2017 schreiber2018/cnn schreiber2018/fcn schreiber2018/ismir2018 sun2021/default zplane/auftakt_v3
boeck2015/tempodetector2016_default 1.0000 0.5391 0.4810 0.0166 0.0000 0.0000 0.0181 0.0033 0.0000 0.1781 0.0000 0.0021 0.1356 0.0040 0.5395 0.6931 0.0040 0.0000 0.0000
boeck2019/multi_task 0.5391 1.0000 0.7787 0.0034 0.0000 0.0000 0.0992 0.0160 0.0000 0.4864 0.0000 0.0078 0.3663 0.0021 0.9942 0.8443 0.0283 0.0001 0.0000
boeck2019/multi_task_hjdb 0.4810 0.7787 1.0000 0.0024 0.0000 0.0000 0.1182 0.0220 0.0000 0.5459 0.0000 0.0090 0.3983 0.0010 0.9308 0.7686 0.0319 0.0002 0.0000
boeck2020/dar 0.0166 0.0034 0.0024 1.0000 0.0000 0.0000 0.0002 0.0000 0.0000 0.0009 0.0000 0.0000 0.0005 0.4784 0.0159 0.0180 0.0000 0.0000 0.0000
davies2009/mirex_qm_tempotracker 0.0000 0.0000 0.0000 0.0000 1.0000 0.0125 0.0021 0.0141 0.0056 0.0000 0.0000 0.0268 0.0001 0.0000 0.0000 0.0000 0.0197 0.1279 0.0439
echonest/version_3_2_1 0.0000 0.0000 0.0000 0.0000 0.0125 1.0000 0.0000 0.0000 0.9561 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.6915
gkiokas2012/default 0.0181 0.0992 0.1182 0.0002 0.0021 0.0000 1.0000 0.4601 0.0000 0.3299 0.0000 0.4510 0.4469 0.0000 0.1133 0.0767 0.5366 0.1098 0.0000
klapuri2006/percival2014 0.0033 0.0160 0.0220 0.0000 0.0141 0.0000 0.4601 1.0000 0.0000 0.0646 0.0000 0.9685 0.1169 0.0000 0.0197 0.0057 0.8812 0.4171 0.0000
oliveira2010/ibt 0.0000 0.0000 0.0000 0.0000 0.0056 0.9561 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.6091
percival2014/stem 0.1781 0.4864 0.5459 0.0009 0.0000 0.0000 0.3299 0.0646 0.0000 1.0000 0.0000 0.0664 0.8375 0.0001 0.4215 0.3503 0.0660 0.0026 0.0000
scheirer1998/percival2014 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2014/default 0.0021 0.0078 0.0090 0.0000 0.0268 0.0000 0.4510 0.9685 0.0000 0.0664 0.0000 1.0000 0.0270 0.0000 0.0093 0.0088 0.9116 0.3197 0.0000
schreiber2017/ismir2017 0.1356 0.3663 0.3983 0.0005 0.0001 0.0000 0.4469 0.1169 0.0000 0.8375 0.0000 0.0270 1.0000 0.0000 0.3180 0.2653 0.1112 0.0064 0.0000
schreiber2017/mirex2017 0.0040 0.0021 0.0010 0.4784 0.0000 0.0000 0.0000 0.0000 0.0000 0.0001 0.0000 0.0000 0.0000 1.0000 0.0014 0.0021 0.0000 0.0000 0.0000
schreiber2018/cnn 0.5395 0.9942 0.9308 0.0159 0.0000 0.0000 0.1133 0.0197 0.0000 0.4215 0.0000 0.0093 0.3180 0.0014 1.0000 0.8271 0.0061 0.0006 0.0000
schreiber2018/fcn 0.6931 0.8443 0.7686 0.0180 0.0000 0.0000 0.0767 0.0057 0.0000 0.3503 0.0000 0.0088 0.2653 0.0021 0.8271 1.0000 0.0097 0.0002 0.0000
schreiber2018/ismir2018 0.0040 0.0283 0.0319 0.0000 0.0197 0.0000 0.5366 0.8812 0.0000 0.0660 0.0000 0.9116 0.1112 0.0000 0.0061 0.0097 1.0000 0.2822 0.0001
sun2021/default 0.0000 0.0001 0.0002 0.0000 0.1279 0.0000 0.1098 0.4171 0.0000 0.0026 0.0000 0.3197 0.0064 0.0000 0.0006 0.0002 0.2822 1.0000 0.0005
zplane/auftakt_v3 0.0000 0.0000 0.0000 0.0000 0.0439 0.6915 0.0000 0.0000 0.6091 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0001 0.0005 1.0000

Table 37: Paired t-test p-values, using reference annotations 2.0 as groundtruth with AOE2. H0: the true mean difference between paired samples is zero. If p<=ɑ, reject H0, i.e. we have a significant difference between estimates from the two algorithms. In the table, p-values<0.05 are set in bold.

CSV JSON LATEX PICKLE

Estimator boeck2015/tempodetector2016_default boeck2019/multi_task boeck2019/multi_task_hjdb boeck2020/dar davies2009/mirex_qm_tempotracker echonest/version_3_2_1 gkiokas2012/default klapuri2006/percival2014 oliveira2010/ibt percival2014/stem scheirer1998/percival2014 schreiber2014/default schreiber2017/ismir2017 schreiber2017/mirex2017 schreiber2018/cnn schreiber2018/fcn schreiber2018/ismir2018 sun2021/default zplane/auftakt_v3
boeck2015/tempodetector2016_default 1.0000 0.7400 0.6622 0.0269 0.0000 0.0000 0.0172 0.0095 0.0000 0.5518 0.0000 0.0032 0.1790 0.0048 0.9831 0.8634 0.0089 0.0000 0.0000
boeck2019/multi_task 0.7400 1.0000 0.7429 0.0132 0.0000 0.0000 0.0687 0.0219 0.0000 0.7928 0.0000 0.0036 0.2843 0.0039 0.7518 0.6223 0.0216 0.0002 0.0000
boeck2019/multi_task_hjdb 0.6622 0.7429 1.0000 0.0093 0.0000 0.0000 0.0850 0.0308 0.0000 0.8703 0.0000 0.0043 0.3149 0.0019 0.6830 0.5384 0.0249 0.0002 0.0000
boeck2020/dar 0.0269 0.0132 0.0093 1.0000 0.0000 0.0000 0.0003 0.0000 0.0000 0.0131 0.0000 0.0000 0.0010 0.3253 0.0895 0.1014 0.0000 0.0000 0.0000
davies2009/mirex_qm_tempotracker 0.0000 0.0000 0.0000 0.0000 1.0000 0.0106 0.0047 0.0080 0.0117 0.0000 0.0000 0.0427 0.0002 0.0000 0.0000 0.0000 0.0211 0.1220 0.0492
echonest/version_3_2_1 0.0000 0.0000 0.0000 0.0000 0.0106 1.0000 0.0000 0.0000 0.7786 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.5657
gkiokas2012/default 0.0172 0.0687 0.0850 0.0003 0.0047 0.0000 1.0000 0.6854 0.0000 0.1009 0.0000 0.4835 0.4056 0.0000 0.0360 0.0235 0.6609 0.1578 0.0000
klapuri2006/percival2014 0.0095 0.0219 0.0308 0.0000 0.0080 0.0000 0.6854 1.0000 0.0000 0.0313 0.0000 0.7636 0.1881 0.0000 0.0103 0.0028 0.9669 0.3014 0.0000
oliveira2010/ibt 0.0000 0.0000 0.0000 0.0000 0.0117 0.7786 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0002 0.7299
percival2014/stem 0.5518 0.7928 0.8703 0.0131 0.0000 0.0000 0.1009 0.0313 0.0000 1.0000 0.0000 0.0123 0.4247 0.0010 0.4895 0.4342 0.0178 0.0004 0.0000
scheirer1998/percival2014 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
schreiber2014/default 0.0032 0.0036 0.0043 0.0000 0.0427 0.0000 0.4835 0.7636 0.0000 0.0123 0.0000 1.0000 0.0227 0.0000 0.0013 0.0016 0.7816 0.4201 0.0001
schreiber2017/ismir2017 0.1790 0.2843 0.3149 0.0010 0.0002 0.0000 0.4056 0.1881 0.0000 0.4247 0.0000 0.0227 1.0000 0.0000 0.1152 0.1087 0.1324 0.0089 0.0000
schreiber2017/mirex2017 0.0048 0.0039 0.0019 0.3253 0.0000 0.0000 0.0000 0.0000 0.0000 0.0010 0.0000 0.0000 0.0000 1.0000 0.0059 0.0079 0.0000 0.0000 0.0000
schreiber2018/cnn 0.9831 0.7518 0.6830 0.0895 0.0000 0.0000 0.0360 0.0103 0.0000 0.4895 0.0000 0.0013 0.1152 0.0059 1.0000 0.8520 0.0018 0.0001 0.0000
schreiber2018/fcn 0.8634 0.6223 0.5384 0.1014 0.0000 0.0000 0.0235 0.0028 0.0000 0.4342 0.0000 0.0016 0.1087 0.0079 0.8520 1.0000 0.0029 0.0001 0.0000
schreiber2018/ismir2018 0.0089 0.0216 0.0249 0.0000 0.0211 0.0000 0.6609 0.9669 0.0000 0.0178 0.0000 0.7816 0.1324 0.0000 0.0018 0.0029 1.0000 0.3021 0.0001
sun2021/default 0.0000 0.0002 0.0002 0.0000 0.1220 0.0000 0.1578 0.3014 0.0002 0.0004 0.0000 0.4201 0.0089 0.0000 0.0001 0.0001 0.3021 1.0000 0.0007
zplane/auftakt_v3 0.0000 0.0000 0.0000 0.0000 0.0492 0.5657 0.0000 0.0000 0.7299 0.0000 0.0000 0.0001 0.0000 0.0000 0.0000 0.0000 0.0001 0.0007 1.0000

Table 38: Paired t-test p-values, using reference annotations 1.0 as groundtruth with AOE2. H0: the true mean difference between paired samples is zero. If p<=ɑ, reject H0, i.e. we have a significant difference between estimates from the two algorithms. In the table, p-values<0.05 are set in bold.

CSV JSON LATEX PICKLE

AOE1 on cvar-Subsets

How well does an estimator perform, when only taking tracks into account that have a cvar-value of less than τ, i.e., have a more or less stable beat?

AOE1 on cvar-Subsets for 1.0 based on cvar-Values from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28

Figure 110: Mean AOE1 compared to version 1.0 for tracks with cvar < τ based on beat annotations from 1.0.

CSV JSON LATEX PICKLE SVG PDF PNG

AOE1 on cvar-Subsets for 2.0 based on cvar-Values from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28

Figure 111: Mean AOE1 compared to version 2.0 for tracks with cvar < τ based on beat annotations from 2.0.

CSV JSON LATEX PICKLE SVG PDF PNG

AOE1 on cvar-Subsets for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI based on cvar-Values from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28

Figure 112: Mean AOE1 compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI for tracks with cvar < τ based on beat annotations from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI.

CSV JSON LATEX PICKLE SVG PDF PNG

AOE1 on cvar-Subsets for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI based on cvar-Values from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28

Figure 113: Mean AOE1 compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI for tracks with cvar < τ based on beat annotations from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI.

CSV JSON LATEX PICKLE SVG PDF PNG

AOE2 on cvar-Subsets

How well does an estimator perform, when only taking tracks into account that have a cvar-value of less than τ, i.e., have a more or less stable beat?

AOE2 on cvar-Subsets for 1.0 based on cvar-Values from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28

Figure 114: Mean AOE2 compared to version 1.0 for tracks with cvar < τ based on beat annotations from 1.0.

CSV JSON LATEX PICKLE SVG PDF PNG

AOE2 on cvar-Subsets for 2.0 based on cvar-Values from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28

Figure 115: Mean AOE2 compared to version 2.0 for tracks with cvar < τ based on beat annotations from 2.0.

CSV JSON LATEX PICKLE SVG PDF PNG

AOE2 on cvar-Subsets for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI based on cvar-Values from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28

Figure 116: Mean AOE2 compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI for tracks with cvar < τ based on beat annotations from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI.

CSV JSON LATEX PICKLE SVG PDF PNG

AOE2 on cvar-Subsets for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI based on cvar-Values from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28

Figure 117: Mean AOE2 compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI for tracks with cvar < τ based on beat annotations from GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI.

CSV JSON LATEX PICKLE SVG PDF PNG

AOE1 on Tempo-Subsets

How well does an estimator perform, when only taking a subset of the reference annotations into account? The graphs show mean AOE1 for reference subsets with tempi in [T-10,T+10] BPM. Note that the graphs do not show confidence intervals and that some values may be based on very few estimates.

AOE1 on Tempo-Subsets for 1.0

Figure 118: Mean AOE1 for estimates compared to version 1.0 for tempo intervals around T.

CSV JSON LATEX PICKLE SVG PDF PNG

AOE1 on Tempo-Subsets for 2.0

Figure 119: Mean AOE1 for estimates compared to version 2.0 for tempo intervals around T.

CSV JSON LATEX PICKLE SVG PDF PNG

AOE1 on Tempo-Subsets for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI

Figure 120: Mean AOE1 for estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI for tempo intervals around T.

CSV JSON LATEX PICKLE SVG PDF PNG

AOE1 on Tempo-Subsets for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI

Figure 121: Mean AOE1 for estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI for tempo intervals around T.

CSV JSON LATEX PICKLE SVG PDF PNG

AOE2 on Tempo-Subsets

How well does an estimator perform, when only taking a subset of the reference annotations into account? The graphs show mean AOE2 for reference subsets with tempi in [T-10,T+10] BPM. Note that the graphs do not show confidence intervals and that some values may be based on very few estimates.

AOE2 on Tempo-Subsets for 1.0

Figure 122: Mean AOE2 for estimates compared to version 1.0 for tempo intervals around T.

CSV JSON LATEX PICKLE SVG PDF PNG

AOE2 on Tempo-Subsets for 2.0

Figure 123: Mean AOE2 for estimates compared to version 2.0 for tempo intervals around T.

CSV JSON LATEX PICKLE SVG PDF PNG

AOE2 on Tempo-Subsets for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI

Figure 124: Mean AOE2 for estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI for tempo intervals around T.

CSV JSON LATEX PICKLE SVG PDF PNG

AOE2 on Tempo-Subsets for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI

Figure 125: Mean AOE2 for estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI for tempo intervals around T.

CSV JSON LATEX PICKLE SVG PDF PNG

Estimated AOE1 for Tempo

When fitting a generalized additive model (GAM) to AOE1-values and a ground truth, what AOE1 can we expect with confidence?

Estimated AOE1 for Tempo for 1.0

Predictions of GAMs trained on AOE1 for estimates for reference 1.0.

Figure 126: AOE1 predictions of a generalized additive model (GAM) fit to AOE1 results for 1.0. The 95% confidence interval around the prediction is shaded in gray.

CSV JSON LATEX PICKLE SVG PDF PNG

Estimated AOE1 for Tempo for 2.0

Predictions of GAMs trained on AOE1 for estimates for reference 2.0.

Figure 127: AOE1 predictions of a generalized additive model (GAM) fit to AOE1 results for 2.0. The 95% confidence interval around the prediction is shaded in gray.

CSV JSON LATEX PICKLE SVG PDF PNG

Estimated AOE1 for Tempo for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI

Predictions of GAMs trained on AOE1 for estimates for reference GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI.

Figure 128: AOE1 predictions of a generalized additive model (GAM) fit to AOE1 results for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI. The 95% confidence interval around the prediction is shaded in gray.

CSV JSON LATEX PICKLE SVG PDF PNG

Estimated AOE1 for Tempo for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI

Predictions of GAMs trained on AOE1 for estimates for reference GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI.

Figure 129: AOE1 predictions of a generalized additive model (GAM) fit to AOE1 results for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI. The 95% confidence interval around the prediction is shaded in gray.

CSV JSON LATEX PICKLE SVG PDF PNG

Estimated AOE2 for Tempo

When fitting a generalized additive model (GAM) to AOE2-values and a ground truth, what AOE2 can we expect with confidence?

Estimated AOE2 for Tempo for 1.0

Predictions of GAMs trained on AOE2 for estimates for reference 1.0.

Figure 130: AOE2 predictions of a generalized additive model (GAM) fit to AOE2 results for 1.0. The 95% confidence interval around the prediction is shaded in gray.

CSV JSON LATEX PICKLE SVG PDF PNG

Estimated AOE2 for Tempo for 2.0

Predictions of GAMs trained on AOE2 for estimates for reference 2.0.

Figure 131: AOE2 predictions of a generalized additive model (GAM) fit to AOE2 results for 2.0. The 95% confidence interval around the prediction is shaded in gray.

CSV JSON LATEX PICKLE SVG PDF PNG

Estimated AOE2 for Tempo for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI

Predictions of GAMs trained on AOE2 for estimates for reference GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI.

Figure 132: AOE2 predictions of a generalized additive model (GAM) fit to AOE2 results for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI. The 95% confidence interval around the prediction is shaded in gray.

CSV JSON LATEX PICKLE SVG PDF PNG

Estimated AOE2 for Tempo for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI

Predictions of GAMs trained on AOE2 for estimates for reference GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI.

Figure 133: AOE2 predictions of a generalized additive model (GAM) fit to AOE2 results for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI. The 95% confidence interval around the prediction is shaded in gray.

CSV JSON LATEX PICKLE SVG PDF PNG

AOE1 for ‘tag_open’ Tags

How well does an estimator perform, when only taking tracks into account that are tagged with some kind of label? Note that some values may be based on very few estimates.

AOE1 for ‘tag_open’ Tags for 1.0

Figure 134: AOE1 of estimates compared to version 1.0 depending on tag from namespace ‘tag_open’.

SVG PDF PNG

AOE1 for ‘tag_open’ Tags for 2.0

Figure 135: AOE1 of estimates compared to version 2.0 depending on tag from namespace ‘tag_open’.

SVG PDF PNG

AOE1 for ‘tag_open’ Tags for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI

Figure 136: AOE1 of estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI depending on tag from namespace ‘tag_open’.

SVG PDF PNG

AOE1 for ‘tag_open’ Tags for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI

Figure 137: AOE1 of estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI depending on tag from namespace ‘tag_open’.

SVG PDF PNG

AOE1 for ‘tag_gtzan’ Tags

How well does an estimator perform, when only taking tracks into account that are tagged with some kind of label? Note that some values may be based on very few estimates.

AOE1 for ‘tag_gtzan’ Tags for 1.0

Figure 138: AOE1 of estimates compared to version 1.0 depending on tag from namespace ‘tag_gtzan’.

SVG PDF PNG

AOE1 for ‘tag_gtzan’ Tags for 2.0

Figure 139: AOE1 of estimates compared to version 2.0 depending on tag from namespace ‘tag_gtzan’.

SVG PDF PNG

AOE1 for ‘tag_gtzan’ Tags for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI

Figure 140: AOE1 of estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI depending on tag from namespace ‘tag_gtzan’.

SVG PDF PNG

AOE1 for ‘tag_gtzan’ Tags for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI

Figure 141: AOE1 of estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI depending on tag from namespace ‘tag_gtzan’.

SVG PDF PNG

AOE2 for ‘tag_open’ Tags

How well does an estimator perform, when only taking tracks into account that are tagged with some kind of label? Note that some values may be based on very few estimates.

AOE2 for ‘tag_open’ Tags for 1.0

Figure 142: AOE2 of estimates compared to version 1.0 depending on tag from namespace ‘tag_open’.

SVG PDF PNG

AOE2 for ‘tag_open’ Tags for 2.0

Figure 143: AOE2 of estimates compared to version 2.0 depending on tag from namespace ‘tag_open’.

SVG PDF PNG

AOE2 for ‘tag_open’ Tags for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI

Figure 144: AOE2 of estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI depending on tag from namespace ‘tag_open’.

SVG PDF PNG

AOE2 for ‘tag_open’ Tags for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI

Figure 145: AOE2 of estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI depending on tag from namespace ‘tag_open’.

SVG PDF PNG

AOE2 for ‘tag_gtzan’ Tags

How well does an estimator perform, when only taking tracks into account that are tagged with some kind of label? Note that some values may be based on very few estimates.

AOE2 for ‘tag_gtzan’ Tags for 1.0

Figure 146: AOE2 of estimates compared to version 1.0 depending on tag from namespace ‘tag_gtzan’.

SVG PDF PNG

AOE2 for ‘tag_gtzan’ Tags for 2.0

Figure 147: AOE2 of estimates compared to version 2.0 depending on tag from namespace ‘tag_gtzan’.

SVG PDF PNG

AOE2 for ‘tag_gtzan’ Tags for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI

Figure 148: AOE2 of estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_IBI depending on tag from namespace ‘tag_gtzan’.

SVG PDF PNG

AOE2 for ‘tag_gtzan’ Tags for GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI

Figure 149: AOE2 of estimates compared to version GTZAN-Rhythm_v2_ismir2015_lbd_2015-10-28_ICBI depending on tag from namespace ‘tag_gtzan’.

SVG PDF PNG

Generated by tempo_eval 0.1.1 on 2022-06-29 18:39. Size L.