Update results
This commit is contained in:
157
Friedman.md
157
Friedman.md
@@ -1,76 +1,83 @@
|
||||
```bash
|
||||
Best results for accuracy as of 2024-12-23 20:12
|
||||
------------------------------------------------
|
||||
# Dataset Odte TBRRoF TBRaF TBRoF pbc4cip
|
||||
=== ============================== ============ ============ ============ ============ ============
|
||||
0 balance-scale 0.9750428259 0.9044194023 0.9023101421 0.7786507839 0.9067200000
|
||||
1 balloons 0.7250000060 0.6287500000 0.6000000000 0.6187500000 0.6900000000
|
||||
2 breast-cancer 0.7516938877 0.7347472506 0.7331207795 0.7227416554 0.6195160000
|
||||
3 breast-cancer-wisc 0.9706747293 0.9688476524 0.9690400351 0.9618093707 0.9703870000
|
||||
4 breast-cancer-wisc-diag 0.9761007452 0.9740889392 0.9713513247 0.9734280508 0.9632700000
|
||||
5 breast-cancer-wisc-prog 0.8031222689 0.8114785914 0.7998559424 0.8043037215 0.6115130000
|
||||
6 cardiotocography-10clases 0.8404853785 0.8326822555 0.8324371518 0.7827072711 0.8685810000
|
||||
7 cardiotocography-3clases 0.9263860846 0.9222234235 0.9196784360 0.8954016281 0.9238460000
|
||||
8 conn-bench-sonar-mines-rocks 0.8586967516 0.8356730769 0.8216346154 0.7904807692 0.8351570000
|
||||
9 cylinder-bands 0.7841918910 0.7728125000 0.7510156250 0.7148437500 0.7812320000
|
||||
10 dermatology 0.9805705738 0.9751872858 0.9741096538 0.9731673166 0.9707740000
|
||||
11 echocardiogram 0.8550358975 0.8506562500 0.8496339286 0.8291785714 0.7800000000
|
||||
12 fertility 0.8806014967 0.8810000000 0.8804000000 0.8800000000 0.6690000000
|
||||
13 haberman-survival 0.7349814928 0.7373734818 0.7355786100 0.7325691633 0.6434060000
|
||||
14 heart-hungarian 0.8231630635 0.8183260274 0.8203068493 0.8067561644 0.8149330000
|
||||
15 hepatitis 0.8424690831 0.8180006418 0.8194833119 0.8099101412 0.7729030000
|
||||
16 ilpd-indian-liver 0.7315809560 0.7132868127 0.7142758621 0.7072269338 0.6763590000
|
||||
17 ionosphere 0.9498350000 0.9471168582 0.9320804598 0.9069444444 0.9378910000
|
||||
18 iris 0.9653333235 0.9653153153 0.9629868330 0.9771552322 0.9653330000
|
||||
19 led-display 0.7274587321 0.6861600000 0.7126200000 0.7028800000 0.7287000000
|
||||
20 libras 0.8726855135 0.8851111111 0.8527222222 0.7556111111 0.8383330000
|
||||
21 low-res-spect 0.9115328133 0.8993678451 0.9006565657 0.8686498316 0.8913450000
|
||||
22 lymphography 0.8636181998 0.8559459459 0.8485135135 0.7733783784 0.8409660000
|
||||
23 mammographic 0.8352833545 0.8260945712 0.8295917185 0.8129815871 0.8185240000
|
||||
24 molec-biol-promoter 0.9013679683 0.8146428571 0.8230357143 0.7479807692 0.8785710000
|
||||
25 musk-1 0.9331642866 0.8939075630 0.8807142857 0.8628991597 0.8922350000
|
||||
26 oocytes_merluccius_nucleus_4d 0.8410942197 0.8388691539 0.8391387808 0.8256020447 0.8326920000
|
||||
27 oocytes_merluccius_states_2f 0.9325797427 0.9270210575 0.9238093385 0.9195349050 0.9045020000
|
||||
28 oocytes_trisopterus_nucleus_2f 0.8557010734 0.8365350877 0.8345175439 0.8048684211 0.8132820000
|
||||
29 oocytes_trisopterus_states_5b 0.9309143639 0.9323026316 0.9278508772 0.9303508772 0.9210510000
|
||||
30 parkinsons 0.9344102585 0.9310110294 0.9143321078 0.8947732843 0.8600000000
|
||||
31 pima 0.7726720953 0.7685416667 0.7633072917 0.7675260417 0.7462240000
|
||||
32 pittsburg-bridges-MATERIAL 0.8731722212 0.8602335165 0.8666620879 0.8014423077 0.7973590000
|
||||
33 pittsburg-bridges-REL-L 0.7060952485 0.6904285714 0.6955642857 0.6185142857 0.6720950000
|
||||
34 pittsburg-bridges-SPAN 0.6941609919 0.6923913043 0.6865217391 0.6221739130 0.6247370000
|
||||
35 pittsburg-bridges-T-OR-D 0.8686441112 0.8702074074 0.8649703704 0.8714222222 0.7877140000
|
||||
36 planning 0.7309008932 0.7202198582 0.7145531915 0.7131205674 0.4685740000
|
||||
37 post-operative 0.7120519483 0.7108901515 0.7074242424 0.7114583333 0.3988890000
|
||||
38 seeds 0.9604761744 0.9467770655 0.9438782051 0.9506552707 0.9376190000
|
||||
39 statlog-australian-credit 0.6782657218 0.6782701818 0.6782668404 0.6782691794 0.5549280000
|
||||
40 statlog-german-credit 0.7681999898 0.7577600000 0.7548000000 0.7560800000 0.7245000000
|
||||
41 statlog-heart 0.8481481481 0.8367207441 0.8373837335 0.8249686351 0.8285190000
|
||||
42 statlog-image 0.9711255431 0.9759832138 0.9697836166 0.9568146987 0.9755840000
|
||||
43 statlog-vehicle 0.8132791913 0.7862973099 0.7764671918 0.8032546781 0.7786160000
|
||||
44 synthetic-control 0.9910000086 0.9919666667 0.9913000000 0.9807000000 0.9861670000
|
||||
45 tic-tac-toe 0.9939462876 0.9856599073 0.9799358496 0.9832547440 0.9867430000
|
||||
46 vertebral-column-2clases 0.8635483825 0.8504972875 0.8500682229 0.8505170146 0.8222580000
|
||||
47 wine 0.9843730736 0.9826086957 0.9816847826 0.9878903162 0.9702860000
|
||||
48 zoo 0.9706897533 0.9465538462 0.9410846154 0.9461615385 0.9742860000
|
||||
=== ============================== ============ ============ ============ ============ ============
|
||||
Average....................... 0.85950053 0.84634616 0.84245834 0.82428080 0.80930857
|
||||
<!-- This file has been generated by the platform program
|
||||
Date: 2024-12-27 11:35
|
||||
|
||||
***************************************************************************************************************
|
||||
Friedman test: H0: 'There is no significant differences between all the classifiers.'
|
||||
Friedman statistic: 87,6408
|
||||
Critical χ2 Value for df=4 and alpha=0,05: 9,4877290e+00
|
||||
p-value: 4,1741784e-18 is less than 0,05
|
||||
The null hypothesis H0 is rejected.
|
||||
***************************************************************************************************************
|
||||
*************************************************************************************************************
|
||||
Post-hoc Holm test: H0: 'There is no significant differences between the control model and the other models.'
|
||||
Control model: Odte
|
||||
Model p-value rank win tie loss Status
|
||||
======= ============ ========= === === ==== =============
|
||||
Odte 1,4285715
|
||||
TBRRoF 1,401302e-03 2,4489796 40 0 9 ✗ rejected H0
|
||||
TBRaF 3,816278e-09 3,3469388 46 0 3 ✗ rejected H0
|
||||
pbc4cip 8,726353e-14 3,8571429 45 0 4 ✗ rejected H0
|
||||
TBRoF 2,575717e-14 3,9183674 44 0 5 ✗ rejected H0
|
||||
*************************************************************************************************************
|
||||
```
|
||||
Table of results
|
||||
-->
|
||||
| # | Dataset | BaggingJ48-SVMODT | BaggingWodt | Odte | PBC4cip | RandomForest | TBRRoF | TBRaF | TBRoF | XGBoost |
|
||||
|--: | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
|
||||
| 1 | balance-scale | 0.9498±0.018 | **0.9566±0.018** | 0.9238±0.022 | 0.9067±0.025 | 0.8376±0.020 | 0.9028±0.021 | 0.9023±0.024 | 0.6981±0.231 | 0.8632±0.026 |
|
||||
| 2 | balloons | 0.5983±0.219 | 0.6700±0.247 | 0.6450±0.304 | **0.6900±0.271** | 0.6233±0.285 | 0.6388±0.236 | 0.5938±0.230 | 0.5837±0.241 | 0.6300±0.305 |
|
||||
| 3 | breast-cancer-wisc-diag | 0.9652±0.018 | **0.9768±0.012** | 0.9717±0.015 | 0.9633±0.014 | 0.9597±0.018 | 0.9724±0.014 | 0.9718±0.013 | 0.9575±0.016 | 0.9645±0.019 |
|
||||
| 4 | breast-cancer-wisc-prog | **0.8039±0.061** | 0.7991±0.051 | 0.7947±0.049 | 0.6115±0.066 | 0.7936±0.030 | 0.8007±0.058 | 0.7880±0.061 | 0.7515±0.060 | 0.7929±0.045 |
|
||||
| 5 | breast-cancer-wisc | 0.9685±0.012 | 0.9631±0.012 | 0.9671±0.012 | **0.9704±0.013** | 0.9680±0.012 | 0.9679±0.014 | 0.9682±0.012 | 0.9404±0.016 | 0.9575±0.014 |
|
||||
| 6 | breast-cancer | 0.7227±0.051 | 0.7026±0.053 | **0.7391±0.026** | 0.6195±0.072 | 0.7279±0.046 | 0.7358±0.040 | 0.7378±0.046 | 0.6895±0.050 | 0.6689±0.054 |
|
||||
| 7 | cardiotocography-10clases | 0.8692±0.014 | 0.8344±0.020 | 0.8378±0.017 | 0.8686±0.016 | 0.8714±0.018 | 0.6963±0.025 | 0.6568±0.058 | 0.7837±0.018 | **0.8841±0.015** |
|
||||
| 8 | cardiotocography-3clases | 0.9442±0.011 | 0.9235±0.013 | 0.9149±0.013 | 0.9238±0.012 | 0.9428±0.011 | 0.8594±0.019 | 0.8737±0.018 | 0.8930±0.014 | **0.9518±0.008** |
|
||||
| 9 | conn-bench-sonar-mines-rocks | 0.7952±0.071 | **0.8469±0.057** | 0.7779±0.060 | 0.8352±0.056 | 0.8117±0.053 | 0.8378±0.047 | 0.8213±0.051 | 0.7517±0.053 | 0.8413±0.055 |
|
||||
| 10 | cylinder-bands | 0.7895±0.036 | 0.7605±0.045 | 0.7441±0.043 | 0.7812±0.043 | 0.7941±0.039 | 0.7591±0.039 | 0.7338±0.045 | 0.6790±0.043 | **0.7992±0.031** |
|
||||
| 11 | dermatology | 0.9667±0.025 | 0.9686±0.018 | 0.9702±0.016 | 0.9708±0.017 | 0.9751±0.014 | **0.9755±0.015** | 0.9742±0.015 | 0.9664±0.018 | 0.9659±0.019 |
|
||||
| 12 | echocardiogram | 0.8374±0.065 | 0.7722±0.074 | 0.8306±0.052 | 0.7800±0.070 | 0.8208±0.062 | **0.8425±0.056** | 0.8415±0.055 | 0.7448±0.070 | 0.8033±0.074 |
|
||||
| 13 | fertility | 0.8720±0.069 | 0.8570±0.056 | 0.8644±0.035 | 0.6690±0.089 | 0.8710±0.038 | **0.8800±0.060** | **0.8800±0.059** | 0.7890±0.088 | 0.8430±0.052 |
|
||||
| 14 | haberman-survival | 0.7244±0.053 | 0.6807±0.052 | 0.7301±0.016 | 0.6434±0.050 | 0.6968±0.049 | 0.7291±0.042 | **0.7305±0.043** | 0.7252±0.046 | 0.6599±0.048 |
|
||||
| 15 | heart-hungarian | 0.8000±0.052 | 0.8112±0.049 | 0.8157±0.045 | 0.8149±0.051 | 0.8130±0.039 | 0.8138±0.038 | **0.8179±0.041** | 0.7598±0.045 | 0.7939±0.048 |
|
||||
| 16 | hepatitis | 0.7948±0.071 | 0.8239±0.086 | 0.8171±0.065 | 0.7729±0.075 | **0.8311±0.054** | 0.8139±0.050 | 0.8134±0.057 | 0.7618±0.062 | 0.7858±0.058 |
|
||||
| 17 | ilpd-indian-liver | 0.6916±0.037 | 0.7091±0.031 | 0.7136±0.008 | 0.6764±0.041 | 0.7032±0.038 | **0.7137±0.034** | 0.7078±0.031 | 0.6771±0.038 | 0.6938±0.039 |
|
||||
| 18 | ionosphere | 0.9270±0.029 | 0.9120±0.034 | 0.9054±0.030 | **0.9379±0.026** | 0.9325±0.024 | 0.9178±0.025 | 0.9320±0.024 | 0.8642±0.033 | 0.9231±0.028 |
|
||||
| 19 | iris | 0.9433±0.037 | 0.9487±0.031 | 0.9593±0.041 | 0.9653±0.033 | 0.9473±0.040 | 0.9656±0.027 | **0.9665±0.025** | 0.9584±0.027 | 0.9413±0.037 |
|
||||
| 20 | led-display | 0.7209±0.028 | 0.7071±0.030 | 0.7146±0.027 | **0.7287±0.024** | 0.7114±0.026 | 0.3765±0.090 | 0.7044±0.030 | 0.6682±0.102 | 0.7150±0.026 |
|
||||
| 21 | libras | 0.7586±0.051 | 0.8314±0.041 | 0.8729±0.041 | 0.8383±0.045 | 0.8206±0.039 | **0.8839±0.035** | 0.8578±0.040 | 0.7007±0.052 | 0.7547±0.055 |
|
||||
| 22 | low-res-spect | 0.8787±0.033 | 0.8874±0.030 | **0.9111±0.023** | 0.8913±0.027 | 0.8952±0.025 | 0.8993±0.025 | 0.8975±0.024 | 0.8161±0.030 | 0.8891±0.022 |
|
||||
| 23 | lymphography | 0.8062±0.076 | **0.8520±0.058** | 0.8334±0.066 | 0.8410±0.054 | 0.8451±0.053 | 0.8332±0.059 | 0.8331±0.063 | 0.7612±0.071 | 0.8462±0.058 |
|
||||
| 24 | mammographic | 0.8279±0.023 | 0.7827±0.023 | **0.8311±0.026** | 0.8185±0.021 | 0.7997±0.030 | 0.8210±0.022 | 0.8291±0.022 | 0.8120±0.022 | 0.7983±0.027 |
|
||||
| 25 | molec-biol-promoter | 0.8374±0.079 | 0.8180±0.083 | 0.7864±0.081 | 0.8786±0.076 | **0.8872±0.067** | 0.8194±0.075 | 0.8137±0.073 | 0.6292±0.083 | 0.8605±0.073 |
|
||||
| 26 | musk-1 | 0.8948±0.029 | **0.8958±0.036** | 0.8611±0.039 | 0.8922±0.032 | 0.8939±0.032 | 0.8937±0.026 | 0.8780±0.032 | 0.8129±0.036 | 0.8897±0.031 |
|
||||
| 27 | oocytes_merluccius_nucleus_4d | 0.8047±0.024 | 0.8024±0.021 | 0.8079±0.022 | **0.8327±0.023** | 0.8047±0.021 | 0.8251±0.023 | 0.8179±0.022 | 0.8174±0.022 | 0.8115±0.025 |
|
||||
| 28 | oocytes_merluccius_states_2f | 0.9172±0.021 | 0.9189±0.017 | 0.9242±0.013 | 0.9045±0.016 | 0.9242±0.018 | 0.9237±0.015 | 0.9226±0.015 | 0.9165±0.017 | **0.9281±0.018** |
|
||||
| 29 | oocytes_trisopterus_nucleus_2f | **0.8317±0.027** | 0.8086±0.024 | 0.8087±0.024 | 0.8133±0.029 | 0.8102±0.023 | 0.8302±0.024 | 0.8271±0.025 | 0.7843±0.030 | 0.8248±0.028 |
|
||||
| 30 | oocytes_trisopterus_states_5b | 0.9246±0.018 | 0.9236±0.017 | 0.9274±0.017 | 0.9211±0.018 | 0.9145±0.020 | 0.9046±0.019 | 0.9057±0.019 | 0.9277±0.015 | **0.9282±0.017** |
|
||||
| 31 | parkinsons | 0.9021±0.040 | 0.9277±0.035 | 0.9031±0.050 | 0.8600±0.065 | 0.9047±0.051 | **0.9297±0.036** | 0.9141±0.036 | 0.8735±0.045 | 0.9138±0.043 |
|
||||
| 32 | pima | 0.7624±0.032 | 0.7459±0.031 | **0.7725±0.025** | 0.7462±0.032 | 0.7626±0.023 | 0.7655±0.028 | 0.7646±0.029 | 0.7358±0.030 | 0.7332±0.030 |
|
||||
| 33 | pittsburg-bridges-MATERIAL | 0.8530±0.067 | 0.8101±0.074 | 0.8372±0.050 | 0.7974±0.073 | 0.8373±0.052 | 0.8527±0.053 | **0.8651±0.058** | 0.7952±0.079 | 0.8320±0.058 |
|
||||
| 34 | pittsburg-bridges-REL-L | 0.6740±0.101 | 0.6690±0.088 | 0.6851±0.081 | 0.6721±0.115 | **0.6945±0.091** | 0.6779±0.084 | 0.6899±0.078 | 0.5911±0.097 | 0.6592±0.101 |
|
||||
| 35 | pittsburg-bridges-SPAN | 0.6540±0.113 | 0.6481±0.112 | **0.6854±0.089** | 0.6247±0.105 | 0.6420±0.080 | 0.6570±0.089 | 0.6770±0.079 | 0.5811±0.091 | 0.6673±0.091 |
|
||||
| 36 | pittsburg-bridges-T-OR-D | 0.8474±0.076 | 0.8579±0.078 | 0.8638±0.051 | 0.7877±0.087 | 0.8501±0.045 | 0.8628±0.056 | **0.8671±0.064** | 0.8264±0.075 | 0.8502±0.050 |
|
||||
| 37 | planning | 0.7037±0.056 | 0.6696±0.072 | 0.7144±0.010 | 0.4686±0.070 | 0.6848±0.043 | **0.7177±0.057** | 0.7139±0.060 | 0.6137±0.074 | 0.6169±0.070 |
|
||||
| 38 | post-operative | 0.6767±0.081 | 0.5944±0.087 | 0.7036±0.054 | 0.3989±0.100 | 0.6278±0.094 | 0.6920±0.085 | **0.7072±0.084** | 0.5345±0.101 | 0.5689±0.085 |
|
||||
| 39 | seeds | 0.9281±0.044 | 0.9400±0.041 | 0.9357±0.028 | 0.9376±0.031 | 0.9252±0.034 | **0.9478±0.029** | 0.9424±0.030 | 0.9417±0.031 | 0.9348±0.038 |
|
||||
| 40 | statlog-australian-credit | 0.6765±0.035 | 0.6170±0.034 | **0.6783±0.003** | 0.5549±0.036 | 0.6702±0.024 | 0.6454±0.030 | 0.6688±0.030 | 0.5800±0.040 | 0.6336±0.032 |
|
||||
| 41 | statlog-german-credit | **0.7669±0.022** | 0.7597±0.027 | 0.7605±0.018 | 0.7245±0.027 | 0.7642±0.021 | 0.7468±0.026 | 0.7407±0.030 | 0.7263±0.025 | 0.7602±0.023 |
|
||||
| 42 | statlog-heart | 0.8359±0.043 | 0.8267±0.045 | **0.8389±0.043** | 0.8285±0.042 | 0.8348±0.050 | 0.8333±0.042 | 0.8369±0.039 | 0.7514±0.054 | 0.8085±0.047 |
|
||||
| 43 | statlog-image | 0.9774±0.006 | 0.9713±0.007 | 0.9639±0.008 | 0.9756±0.007 | 0.9784±0.008 | 0.9340±0.024 | 0.9420±0.021 | 0.9425±0.070 | **0.9819±0.007** |
|
||||
| 44 | statlog-vehicle | 0.7596±0.028 | 0.7625±0.028 | **0.8129±0.030** | 0.7786±0.026 | 0.7513±0.026 | 0.7360±0.038 | 0.7547±0.030 | 0.7901±0.027 | 0.7689±0.027 |
|
||||
| 45 | synthetic-control | 0.9613±0.022 | 0.9850±0.010 | 0.9865±0.009 | 0.9862±0.010 | 0.9868±0.009 | **0.9922±0.007** | 0.9903±0.008 | 0.9518±0.019 | 0.9753±0.012 |
|
||||
| 46 | tic-tac-toe | 0.9833±0.008 | 0.9855±0.009 | 0.9833±0.009 | 0.9867±0.010 | **0.9890±0.009** | 0.8797±0.029 | 0.8656±0.025 | 0.9833±0.008 | 0.9888±0.008 |
|
||||
| 47 | vertebral-column-2clases | 0.8548±0.043 | 0.8361±0.043 | **0.8565±0.039** | 0.8223±0.038 | 0.8297±0.036 | 0.8434±0.038 | 0.8426±0.037 | 0.8190±0.041 | 0.8271±0.036 |
|
||||
| 48 | wine | 0.9792±0.020 | 0.9786±0.022 | 0.9747±0.021 | 0.9703±0.033 | 0.9797±0.024 | **0.9846±0.017** | 0.9818±0.020 | 0.9797±0.026 | 0.9635±0.031 |
|
||||
| 49 | zoo | 0.9317±0.047 | 0.9575±0.038 | 0.9579±0.048 | **0.9743±0.035** | 0.9607±0.040 | 0.9468±0.045 | 0.9346±0.056 | 0.9307±0.052 | 0.9585±0.035 |
|
||||
| | **Average Score** | 0.8346±0.100 | 0.8303±0.108 | **0.8391±0.095** | 0.8093±0.138 | 0.8347±0.102 | 0.8261±0.116 | 0.8306±0.099 | 0.7871±0.121 | 0.8256±0.111 |
|
||||
|
||||
### Friedman test: H<sub>0</sub>: There is no significant differences between all the classifiers
|
||||
- Friedman statistic: 70,5129
|
||||
- Critical χ2 Value for df=8 and alpha=0,05: 1,5507313e+01
|
||||
- p-value: 3,8843828e-12 is less than 0,05
|
||||
- The null hypothesis H<sub>0</sub> is rejected.
|
||||
|
||||
|
||||
<!-- This file has been generated by the platform program
|
||||
Date: 2024-12-27 10:55:33
|
||||
|
||||
Post-hoc handler test
|
||||
-->
|
||||
### Post-hoc Holm test: H<sub>0</sub>: There is no significant differences between the control model and the other models
|
||||
|
||||
| classifier | pvalue | rank | win | tie | loss | H<sub>0</sub> |
|
||||
| :-- | --: | --: | --:| --: | --: | :--: |
|
||||
| Odte | - | 3,88 | - | - | - |
|
||||
| TBRRoF | **1,0000e+00** |3,95 | 27 | 0 | 22 |**accepted** |
|
||||
| TBRaF | **1,0000e+00** |4,19 | 27 | 0 | 22 |**accepted** |
|
||||
| RandomForest | **8,5429e-01** |4,47 | 27 | 0 | 22 |**accepted** |
|
||||
| BaggingJ48-SVMODT | **4,8534e-01** |4,73 | 31 | 0 | 18 |**accepted** |
|
||||
| BaggingWodt | **8,2523e-02** |5,20 | 32 | 0 | 17 |**accepted** |
|
||||
| XGBoost | 3,8055e-02 |5,39 | 32 | 0 | 17 |rejected |
|
||||
| PBC4cip | 1,7426e-02 |5,55 | 30 | 0 | 19 |rejected |
|
||||
| TBRoF | 9,1628e-11 |7,63 | 45 | 0 | 4 |rejected |
|
||||
|
Reference in New Issue
Block a user