Doctorado-ML/benchmark
1
0
Fork 0
mirror of https://github.com/Doctorado-ML/benchmark.git synced 2025-08-15 15:35:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

Add github ci and badges

Browse Source 
refactor setup
This commit is contained in:
Ricardo Montañana Gómez
2022-05-11 12:21:55 +02:00
parent d826a65300
commit c7768ad387
5 changed files with 146 additions and 45 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
README.md
View File

@@ -1,3 +1,7 @@
![CI](https://github.com/Doctorado-ML/benchmark/workflows/CI/badge.svg)
[![codecov](https://codecov.io/gh/doctorado-ml/benchmark/branch/master/graph/badge.svg)](https://codecov.io/gh/doctorado-ml/benchmark)
[![Language grade: Python](https://img.shields.io/lgtm/grade/python/g/Doctorado-ML/STree.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/Doctorado-ML/STree/context:python)
![https://img.shields.io/badge/python-3.8%2B-blue](https://img.shields.io/badge/python-3.8%2B-brightgreen)
# benchmark
Benchmarking models
@@ -6,53 +10,55 @@ Benchmarking models
```python
# 5 Fold 10 seeds with STree with default hyperparameters and report
python src/main.py -m STree -P iMac27 -r 1
be_main -m STree -P iMac27 -r 1
# Setting number of folds, in this case 7
python src/main.py -m STree -P iMac27 -n 7
be_main -m STree -P iMac27 -n 7
# 5 Fold 10 seeds with STree and best results hyperparams
python src/main.py -m STree -P iMac27 -f 1
be_main -m STree -P iMac27 -f 1
# 5 Fold 10 seeds with STree and same hyperparameters
python src/main.py -m STree -P iMac27 -p '{"kernel": "rbf", "gamma": 0.1}'
be_main -m STree -P iMac27 -p '{"kernel": "rbf", "gamma": 0.1}'
```
## Best Results
```python
# Build best results of STree model and print report
python src/build_best.py -m STree -r 1
be_build_best -m STree -r 1
# Report of STree best results
python src/report.py -b STree
be_report -b STree
```
## Reports
```python
# Datasets list
python src/report.py
be_report
# Report of given experiment
python src/report.py -f results/results_STree_iMac27_2021-09-22_17:13:02.json
be_report -f results/results_STree_iMac27_2021-09-22_17:13:02.json
# Report of given experiment building excel file and compare with best results
python src/report.py -f results/results_STree_iMac27_2021-09-22_17:13:02.json -x 1 -c 1
be_report -f results/results_STree_iMac27_2021-09-22_17:13:02.json -x 1 -c 1
# Report of given experiment building sql file
python src/report.py -f results/results_STree_iMac27_2021-09-22_17:13:02.json -q 1
be_report -f results/results_STree_iMac27_2021-09-22_17:13:02.json -q 1
```
## Benchmark
```python
# Do benchmark and print report
python src/benchmark.py
be_benchmark
# Do benchmark, print report and build excel file with data
python src/benchmark.py -x 1
be_benchmark -x 1
# Do benchmark, print report and build tex table with results
be_benchmark -t 1
```
## List
```python
# List of results of given model
python src/list.py -m ODTE
be_list -m ODTE
# List of results of given model and score
python src/list.py -m STree -s f1-macro
be_list -m STree -s f1-macro
# List all results
python src/list.py
be_list
```