Compare commits
17 Commits
4e18dc87be
...
main
Author | SHA1 | Date | |
---|---|---|---|
c3c580a611
|
|||
515455695b
|
|||
f68d216150
|
|||
b990684581
|
|||
5fd0ef692d
|
|||
dfcdadbf38
|
|||
613f4b6813
|
|||
dc324fe5f7
|
|||
9816896240
|
|||
a3f765ce3c
|
|||
3d814a79c6
|
|||
1ef7ca6180 | |||
9448a971e8
|
|||
24cef7496d
|
|||
a1a6d3d612
|
|||
dda9740e83
|
|||
41afa1b888
|
2
.gitignore
vendored
2
.gitignore
vendored
@@ -43,3 +43,5 @@ diagrams/html/**
|
|||||||
diagrams/latex/**
|
diagrams/latex/**
|
||||||
.cache
|
.cache
|
||||||
vcpkg_installed
|
vcpkg_installed
|
||||||
|
.claude/settings.local.json
|
||||||
|
CMakeUserPresets.json
|
||||||
|
93
CHANGELOG.md
Normal file
93
CHANGELOG.md
Normal file
@@ -0,0 +1,93 @@
|
|||||||
|
# Changelog
|
||||||
|
|
||||||
|
All notable changes to this project will be documented in this file.
|
||||||
|
|
||||||
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
||||||
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||||
|
|
||||||
|
## [Unreleased]
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
- **BREAKING**: Migrated dependency management from vcpkg to Conan
|
||||||
|
- Updated build system to use Conan toolchain files instead of vcpkg
|
||||||
|
- Updated `make init` command to use `conan install` instead of `vcpkg install`
|
||||||
|
- Modified CMakeLists.txt to use Conan's find_package mechanism
|
||||||
|
- Updated documentation in CLAUDE.md to reflect Conan usage
|
||||||
|
|
||||||
|
### Added
|
||||||
|
- `conanfile.py` - Conan recipe for dependency management with all required dependencies
|
||||||
|
- CMakeUserPresets.json (generated by Conan)
|
||||||
|
- Support for Conan build profiles (Release/Debug)
|
||||||
|
|
||||||
|
### Removed
|
||||||
|
- `vcpkg.json` - vcpkg manifest file
|
||||||
|
- `vcpkg-configuration.json` - vcpkg registry configuration
|
||||||
|
- vcpkg toolchain dependency in build system
|
||||||
|
|
||||||
|
### Notes
|
||||||
|
- The migration maintains compatibility with existing make targets and workflow
|
||||||
|
- All dependencies now managed through Conan package manager
|
||||||
|
|
||||||
|
## [1.1.0] - 2025-07-02
|
||||||
|
|
||||||
|
### Added
|
||||||
|
- **AdaBoost Implementation**: Complete multi-class SAMME AdaBoost classifier with optimization
|
||||||
|
- Optimized AdaBoostPredict with 100 estimators as default
|
||||||
|
- Enhanced predictProbaSample functionality
|
||||||
|
- Full predict_proba support for probabilistic predictions
|
||||||
|
- **Decision Tree Classifier**: New base classifier implementation with comprehensive tests
|
||||||
|
- **XA1DE Model Family**: Extended Averaged One-Dependence Estimators
|
||||||
|
- XA1DE, XBAODE, XSPODE variants with threading support
|
||||||
|
- Complete integration with memory optimization
|
||||||
|
- Prior probability computation in prediction
|
||||||
|
- **Wilcoxon Statistical Test**: Statistical significance testing for model comparison
|
||||||
|
- **Folder Management**: Enhanced file organization with folder parameter support across tools
|
||||||
|
- Added folder parameter to b_best, b_grid, b_main, and b_manage
|
||||||
|
- **vcpkg Integration**: Package management system integration (now migrated to Conan)
|
||||||
|
|
||||||
|
### Enhanced
|
||||||
|
- **Grid Search System**: Complete refactoring with MPI parallelization
|
||||||
|
- Grid experiment functionality with conditional result saving
|
||||||
|
- Fixed smoothing problems and dataset ordering
|
||||||
|
- Enhanced reporting and summary generation
|
||||||
|
- **Excel Reporting**: Advanced Excel export capabilities
|
||||||
|
- ReportExcelCompared class for side-by-side result comparison
|
||||||
|
- Enhanced formatting with colors and fixed headers
|
||||||
|
- Automatic file opening after generation
|
||||||
|
- **Results Management**: Comprehensive result handling and validation
|
||||||
|
- JSON schema validation for result format integrity
|
||||||
|
- Improved console reporting with classification reports
|
||||||
|
- Pagination support for large result sets
|
||||||
|
- **Statistical Analysis**: Enhanced statistical testing and reporting
|
||||||
|
- AUC (Area Under Curve) computation and reporting
|
||||||
|
- Confusion matrix generation and visualization
|
||||||
|
- Classification reports with color coding
|
||||||
|
|
||||||
|
### Performance Improvements
|
||||||
|
- Optimized AdaBoost training and prediction algorithms
|
||||||
|
- Enhanced memory management in XA1DE implementations
|
||||||
|
- Improved discretization algorithms with MDLP integration
|
||||||
|
- Faster ROC-AUC computation for binary classification problems
|
||||||
|
|
||||||
|
### Developer Experience
|
||||||
|
- **Testing Framework**: Comprehensive test suite with Catch2
|
||||||
|
- **Build System**: Streamlined CMake configuration with dependency management
|
||||||
|
- **Documentation**: Enhanced project documentation and build instructions
|
||||||
|
- **Code Quality**: Refactored codebase with improved error handling and logging
|
||||||
|
|
||||||
|
### Bug Fixes
|
||||||
|
- Fixed predict_proba implementations across multiple classifiers
|
||||||
|
- Resolved grid search dataset ordering issues
|
||||||
|
- Fixed Excel report formatting and column width problems
|
||||||
|
- Corrected time output formatting in various tools
|
||||||
|
- Fixed memory leaks and stability issues in model implementations
|
||||||
|
|
||||||
|
## [1.0.0] - 2024-01-09
|
||||||
|
|
||||||
|
### Initial Release
|
||||||
|
- **Core Framework**: Machine learning experimentation platform for Bayesian Networks
|
||||||
|
- **Basic Classifiers**: Initial set of Bayesian network classifiers
|
||||||
|
- **Experiment Management**: Basic experiment orchestration and result storage
|
||||||
|
- **Dataset Support**: ARFF file format support with discretization
|
||||||
|
- **Build System**: CMake-based build system with external library integration
|
||||||
|
- **Command Line Tools**: Initial versions of b_main, b_best, b_list utilities
|
139
CLAUDE.md
Normal file
139
CLAUDE.md
Normal file
@@ -0,0 +1,139 @@
|
|||||||
|
# CLAUDE.md
|
||||||
|
|
||||||
|
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||||
|
|
||||||
|
## Project Overview
|
||||||
|
|
||||||
|
Platform is a C++ machine learning framework for running experiments with Bayesian Networks and other classifiers. It supports both research-focused experimental classifiers and production-ready models through a unified interface.
|
||||||
|
|
||||||
|
## Build System
|
||||||
|
|
||||||
|
The project uses CMake with Make as the primary build system:
|
||||||
|
|
||||||
|
- **Release build**: `make release` (creates `build_Release/` directory)
|
||||||
|
- **Debug build**: `make debug` (creates `build_Debug/` directory with testing and coverage enabled)
|
||||||
|
- **Install binaries**: `make install` (copies executables to `~/bin` by default)
|
||||||
|
- **Clean project**: `make clean` (removes build directories)
|
||||||
|
- **Initialize dependencies**: `make init` (runs conan install for both Release and Debug)
|
||||||
|
|
||||||
|
### Testing
|
||||||
|
|
||||||
|
- **Run tests**: `make test` (builds debug version and runs all tests)
|
||||||
|
- **Coverage report**: `make coverage` (runs tests and generates coverage with gcovr)
|
||||||
|
- **Single test with options**: `make test opt="-s"` (verbose) or `make test opt="-c='Test Name'"` (specific test)
|
||||||
|
|
||||||
|
### Build Targets
|
||||||
|
|
||||||
|
Main executables (built from `src/commands/`):
|
||||||
|
- `b_main`: Main experiment runner
|
||||||
|
- `b_grid`: Grid search over hyperparameters
|
||||||
|
- `b_best`: Best results analysis and comparison
|
||||||
|
- `b_list`: Dataset listing and properties
|
||||||
|
- `b_manage`: Results management interface
|
||||||
|
- `b_results`: Results processing
|
||||||
|
|
||||||
|
## Dependencies
|
||||||
|
|
||||||
|
The project uses Conan for package management with these key dependencies:
|
||||||
|
- **libtorch**: PyTorch C++ backend for tensor operations
|
||||||
|
- **nlohmann_json**: JSON processing
|
||||||
|
- **catch2**: Unit testing framework
|
||||||
|
- **cli11**: Command-line argument parsing (replacement for argparse)
|
||||||
|
|
||||||
|
Custom dependencies (not available in ConanCenter):
|
||||||
|
- **fimdlp**: MDLP discretization library (needs manual integration)
|
||||||
|
- **folding**: Cross-validation utilities (needs manual integration)
|
||||||
|
- **arff-files**: ARFF dataset file handling (needs manual integration)
|
||||||
|
|
||||||
|
External dependencies (managed separately):
|
||||||
|
- **BayesNet**: Core Bayesian network classifiers (from `../lib/`)
|
||||||
|
- **PyClassifiers**: Python classifier wrappers (from `../lib/`)
|
||||||
|
- **MPI**: Message Passing Interface for parallel processing
|
||||||
|
- **Boost**: Python integration and utilities
|
||||||
|
|
||||||
|
**Note**: Some dependencies (fimdlp, folding, arff-files) are not available in ConanCenter and need to be:
|
||||||
|
- Built as custom Conan packages, or
|
||||||
|
- Integrated using CMake FetchContent, or
|
||||||
|
- Built separately and found via find_package
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
### Core Components
|
||||||
|
|
||||||
|
**Experiment Framework** (`src/main/`):
|
||||||
|
- `Experiment.cpp/h`: Main experiment orchestration
|
||||||
|
- `Models.cpp/h`: Classifier factory and registration system
|
||||||
|
- `Scores.cpp/h`: Performance metrics calculation
|
||||||
|
- `HyperParameters.cpp/h`: Parameter management
|
||||||
|
- `ArgumentsExperiment.cpp/h`: Command-line argument handling
|
||||||
|
|
||||||
|
**Data Handling** (`src/common/`):
|
||||||
|
- `Dataset.cpp/h`: Individual dataset representation
|
||||||
|
- `Datasets.cpp/h`: Dataset collection management
|
||||||
|
- `Discretization.cpp/h`: Data discretization utilities
|
||||||
|
|
||||||
|
**Classifiers** (`src/experimental_clfs/`):
|
||||||
|
- `AdaBoost.cpp/h`: Multi-class SAMME AdaBoost implementation
|
||||||
|
- `DecisionTree.cpp/h`: Decision tree base classifier
|
||||||
|
- `XA1DE.cpp/h`: Extended AODE variants
|
||||||
|
- Experimental implementations of Bayesian network classifiers
|
||||||
|
|
||||||
|
**Grid Search** (`src/grid/`):
|
||||||
|
- `GridSearch.cpp/h`: Hyperparameter optimization
|
||||||
|
- `GridExperiment.cpp/h`: Grid search experiment management
|
||||||
|
- Uses MPI for parallel hyperparameter evaluation
|
||||||
|
|
||||||
|
**Results & Reporting** (`src/results/`, `src/reports/`):
|
||||||
|
- JSON-based result storage with schema validation
|
||||||
|
- Excel export capabilities via libxlsxwriter
|
||||||
|
- Console and paginated result display
|
||||||
|
|
||||||
|
### Model Registration System
|
||||||
|
|
||||||
|
The framework uses a factory pattern with automatic registration:
|
||||||
|
- All classifiers inherit from `bayesnet::BaseClassifier`
|
||||||
|
- Registration happens in `src/main/modelRegister.h`
|
||||||
|
- Factory creates instances by string name via `Models::create()`
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
**Environment Configuration** (`.env` file):
|
||||||
|
- `experiment`: Experiment name/type
|
||||||
|
- `n_folds`: Cross-validation folds (default: 5)
|
||||||
|
- `seeds`: Random seeds for reproducibility
|
||||||
|
- `model`: Default classifier name
|
||||||
|
- `score`: Primary evaluation metric
|
||||||
|
- `platform`: System identifier for results
|
||||||
|
|
||||||
|
**Grid Search Configuration**:
|
||||||
|
- `grid_<model_name>_input.json`: Hyperparameter search space
|
||||||
|
- `grid_<model_name>_output.json`: Search results
|
||||||
|
|
||||||
|
## Data Format
|
||||||
|
|
||||||
|
**Dataset Requirements**:
|
||||||
|
- ARFF format files in `datasets/` directory
|
||||||
|
- `all.txt` file listing datasets: `<name>,<class_name>,<real_features>`
|
||||||
|
- Supports both discrete and continuous features
|
||||||
|
- Automatic discretization available via MDLP
|
||||||
|
|
||||||
|
**Experimental Data**:
|
||||||
|
- Results stored in JSON format with versioned schemas
|
||||||
|
- Test data in `tests/data/` for unit testing
|
||||||
|
- Sample datasets: iris, diabetes, ecoli, glass, etc.
|
||||||
|
|
||||||
|
## Development Workflow
|
||||||
|
|
||||||
|
1. **Setup**: Run `make init` to install dependencies via Conan
|
||||||
|
2. **Development**: Use `make debug` for development builds with testing
|
||||||
|
3. **Testing**: Run `make test` after changes
|
||||||
|
4. **Release**: Use `make release` for optimized builds
|
||||||
|
5. **Experiments**: Use `.env` configuration and run `b_main` with appropriate flags
|
||||||
|
|
||||||
|
## Key Features
|
||||||
|
|
||||||
|
- **Multi-threaded**: Uses MPI for parallel grid search and experiments
|
||||||
|
- **Cross-platform**: Supports Linux and macOS via vcpkg
|
||||||
|
- **Extensible**: Easy classifier registration and integration
|
||||||
|
- **Research-focused**: Designed for machine learning experimentation
|
||||||
|
- **Visualization**: DOT graph generation for decision trees and networks
|
@@ -13,10 +13,10 @@ set(CMAKE_CXX_STANDARD 20)
|
|||||||
set(CMAKE_CXX_STANDARD_REQUIRED ON)
|
set(CMAKE_CXX_STANDARD_REQUIRED ON)
|
||||||
set(CMAKE_CXX_EXTENSIONS OFF)
|
set(CMAKE_CXX_EXTENSIONS OFF)
|
||||||
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
|
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
|
||||||
|
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
|
||||||
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${TORCH_CXX_FLAGS}")
|
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${TORCH_CXX_FLAGS}")
|
||||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -pthread")
|
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -pthread")
|
||||||
set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -Ofast")
|
set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -O3")
|
||||||
set(CMAKE_CXX_FLAGS_DEBUG " ${CMAKE_CXX_FLAGS} -fprofile-arcs -ftest-coverage -O0 -g")
|
|
||||||
|
|
||||||
# Options
|
# Options
|
||||||
# -------
|
# -------
|
||||||
@@ -43,8 +43,6 @@ find_package(Boost 1.66.0 REQUIRED COMPONENTS python3 numpy3)
|
|||||||
|
|
||||||
# # Python
|
# # Python
|
||||||
find_package(Python3 REQUIRED COMPONENTS Development)
|
find_package(Python3 REQUIRED COMPONENTS Development)
|
||||||
# # target_include_directories(MyTarget SYSTEM PRIVATE ${Python3_INCLUDE_DIRS})
|
|
||||||
# message("Python_LIBRARIES=${Python_LIBRARIES}")
|
|
||||||
|
|
||||||
# # Boost Python
|
# # Boost Python
|
||||||
# find_package(boost_python${Python3_VERSION_MAJOR}${Python3_VERSION_MINOR} CONFIG REQUIRED COMPONENTS python${Python3_VERSION_MAJOR}${Python3_VERSION_MINOR})
|
# find_package(boost_python${Python3_VERSION_MAJOR}${Python3_VERSION_MINOR} CONFIG REQUIRED COMPONENTS python${Python3_VERSION_MAJOR}${Python3_VERSION_MINOR})
|
||||||
@@ -62,35 +60,16 @@ endif()
|
|||||||
|
|
||||||
# External libraries - dependencies of Platform
|
# External libraries - dependencies of Platform
|
||||||
# ---------------------------------------------
|
# ---------------------------------------------
|
||||||
|
find_package(nlohmann_json CONFIG REQUIRED)
|
||||||
find_library(XLSXWRITER_LIB NAMES libxlsxwriter.dylib libxlsxwriter.so PATHS ${Platform_SOURCE_DIR}/lib/libxlsxwriter/lib)
|
find_package(argparse CONFIG REQUIRED)
|
||||||
# find_path(XLSXWRITER_INCLUDE_DIR xlsxwriter.h)
|
|
||||||
# find_library(XLSXWRITER_LIBRARY xlsxwriter)
|
|
||||||
# message("XLSXWRITER_INCLUDE_DIR=${XLSXWRITER_INCLUDE_DIR}")
|
|
||||||
# message("XLSXWRITER_LIBRARY=${XLSXWRITER_LIBRARY}")
|
|
||||||
find_package(Torch CONFIG REQUIRED)
|
find_package(Torch CONFIG REQUIRED)
|
||||||
|
find_package(arff-files CONFIG REQUIRED)
|
||||||
find_package(fimdlp CONFIG REQUIRED)
|
find_package(fimdlp CONFIG REQUIRED)
|
||||||
find_package(folding CONFIG REQUIRED)
|
find_package(folding CONFIG REQUIRED)
|
||||||
find_package(argparse CONFIG REQUIRED)
|
find_package(bayesnet CONFIG REQUIRED)
|
||||||
find_package(nlohmann_json CONFIG REQUIRED)
|
find_package(pyclassifiers CONFIG REQUIRED)
|
||||||
|
find_package(libxlsxwriter CONFIG REQUIRED)
|
||||||
find_package(Boost REQUIRED COMPONENTS python)
|
find_package(Boost REQUIRED COMPONENTS python)
|
||||||
find_package(arff-files CONFIG REQUIRED)
|
|
||||||
|
|
||||||
# BayesNet
|
|
||||||
find_library(bayesnet NAMES libbayesnet bayesnet libbayesnet.a PATHS ${Platform_SOURCE_DIR}/../lib/lib REQUIRED)
|
|
||||||
find_path(Bayesnet_INCLUDE_DIRS REQUIRED NAMES bayesnet PATHS ${Platform_SOURCE_DIR}/../lib/include)
|
|
||||||
add_library(bayesnet::bayesnet UNKNOWN IMPORTED)
|
|
||||||
set_target_properties(bayesnet::bayesnet PROPERTIES
|
|
||||||
IMPORTED_LOCATION ${bayesnet}
|
|
||||||
INTERFACE_INCLUDE_DIRECTORIES ${Bayesnet_INCLUDE_DIRS})
|
|
||||||
message(STATUS "BayesNet=${bayesnet}")
|
|
||||||
message(STATUS "BayesNet_INCLUDE_DIRS=${Bayesnet_INCLUDE_DIRS}")
|
|
||||||
|
|
||||||
# PyClassifiers
|
|
||||||
find_library(PyClassifiers NAMES libPyClassifiers PyClassifiers libPyClassifiers.a PATHS ${Platform_SOURCE_DIR}/../lib/lib REQUIRED)
|
|
||||||
find_path(PyClassifiers_INCLUDE_DIRS REQUIRED NAMES pyclassifiers PATHS ${Platform_SOURCE_DIR}/../lib/include)
|
|
||||||
message(STATUS "PyClassifiers=${PyClassifiers}")
|
|
||||||
message(STATUS "PyClassifiers_INCLUDE_DIRS=${PyClassifiers_INCLUDE_DIRS}")
|
|
||||||
|
|
||||||
# Subdirectories
|
# Subdirectories
|
||||||
# --------------
|
# --------------
|
||||||
@@ -105,14 +84,16 @@ file(GLOB Platform_SOURCES CONFIGURE_DEPENDS ${Platform_SOURCE_DIR}/src/*.cpp)
|
|||||||
# Testing
|
# Testing
|
||||||
# -------
|
# -------
|
||||||
if (ENABLE_TESTING)
|
if (ENABLE_TESTING)
|
||||||
enable_testing()
|
|
||||||
MESSAGE("Testing enabled")
|
MESSAGE("Testing enabled")
|
||||||
|
set(CMAKE_CXX_FLAGS_DEBUG " ${CMAKE_CXX_FLAGS} -fprofile-arcs -ftest-coverage -O0 -g")
|
||||||
|
enable_testing()
|
||||||
find_package(Catch2 CONFIG REQUIRED)
|
find_package(Catch2 CONFIG REQUIRED)
|
||||||
|
set(CODE_COVERAGE ON)
|
||||||
include(CTest)
|
include(CTest)
|
||||||
add_subdirectory(tests)
|
add_subdirectory(tests)
|
||||||
endif (ENABLE_TESTING)
|
endif (ENABLE_TESTING)
|
||||||
if (CODE_COVERAGE)
|
if (CODE_COVERAGE)
|
||||||
include(CodeCoverage)
|
|
||||||
MESSAGE("Code coverage enabled")
|
MESSAGE("Code coverage enabled")
|
||||||
|
include(CodeCoverage)
|
||||||
SET(GCC_COVERAGE_LINK_FLAGS " ${GCC_COVERAGE_LINK_FLAGS} -lgcov --coverage")
|
SET(GCC_COVERAGE_LINK_FLAGS " ${GCC_COVERAGE_LINK_FLAGS} -lgcov --coverage")
|
||||||
endif (CODE_COVERAGE)
|
endif (CODE_COVERAGE)
|
||||||
|
64
Makefile
64
Makefile
@@ -6,6 +6,13 @@ f_release = build_Release
|
|||||||
f_debug = build_Debug
|
f_debug = build_Debug
|
||||||
app_targets = b_best b_list b_main b_manage b_grid b_results
|
app_targets = b_best b_list b_main b_manage b_grid b_results
|
||||||
test_targets = unit_tests_platform
|
test_targets = unit_tests_platform
|
||||||
|
# Set the number of parallel jobs to the number of available processors minus 7
|
||||||
|
CPUS := $(shell getconf _NPROCESSORS_ONLN 2>/dev/null \
|
||||||
|
|| nproc --all 2>/dev/null \
|
||||||
|
|| sysctl -n hw.ncpu)
|
||||||
|
|
||||||
|
# --- Your desired job count: CPUs – 7, but never less than 1 --------------
|
||||||
|
JOBS := $(shell n=$(CPUS); [ $${n} -gt 7 ] && echo $$((n-7)) || echo 1)
|
||||||
|
|
||||||
define ClearTests
|
define ClearTests
|
||||||
@for t in $(test_targets); do \
|
@for t in $(test_targets); do \
|
||||||
@@ -20,15 +27,36 @@ define ClearTests
|
|||||||
fi ;
|
fi ;
|
||||||
endef
|
endef
|
||||||
|
|
||||||
|
define build_target
|
||||||
|
@echo ">>> Building the project for $(1)..."
|
||||||
|
@if [ -d $(2) ]; then rm -fr $(2); fi
|
||||||
|
@conan install . --build=missing -of $(2) -s build_type=$(1)
|
||||||
|
@cmake -S . -B $(2) -DCMAKE_TOOLCHAIN_FILE=$(2)/build/$(1)/generators/conan_toolchain.cmake -DCMAKE_BUILD_TYPE=$(1) -D$(3)
|
||||||
|
@echo ">>> Will build using $(JOBS) parallel jobs"
|
||||||
|
echo ">>> Done"
|
||||||
|
endef
|
||||||
|
|
||||||
|
define compile_target
|
||||||
|
@echo ">>> Compiling for $(1)..."
|
||||||
|
if [ "$(3)" != "" ]; then \
|
||||||
|
target="-t$(3)"; \
|
||||||
|
else \
|
||||||
|
target=""; \
|
||||||
|
fi
|
||||||
|
@cmake --build $(2) --config $(1) --parallel $(JOBS) $(target)
|
||||||
|
@echo ">>> Done"
|
||||||
|
endef
|
||||||
|
|
||||||
init: ## Initialize the project installing dependencies
|
init: ## Initialize the project installing dependencies
|
||||||
@echo ">>> Installing dependencies"
|
@echo ">>> Installing dependencies with Conan"
|
||||||
@vcpkg install
|
@conan install . --output-folder=build --build=missing -s build_type=Release
|
||||||
@echo ">>> Done";
|
@conan install . --output-folder=build_debug --build=missing -s build_type=Debug
|
||||||
|
@echo ">>> Done"
|
||||||
|
|
||||||
clean: ## Clean the project
|
clean: ## Clean the project
|
||||||
@echo ">>> Cleaning the project..."
|
@echo ">>> Cleaning the project..."
|
||||||
@if test -f CMakeCache.txt ; then echo "- Deleting CMakeCache.txt"; rm -f CMakeCache.txt; fi
|
@if test -f CMakeCache.txt ; then echo "- Deleting CMakeCache.txt"; rm -f CMakeCache.txt; fi
|
||||||
@for folder in $(f_release) $(f_debug) vpcpkg_installed install_test ; do \
|
@for folder in $(f_release) $(f_debug) build build_debug install_test ; do \
|
||||||
if test -d "$$folder" ; then \
|
if test -d "$$folder" ; then \
|
||||||
echo "- Deleting $$folder folder" ; \
|
echo "- Deleting $$folder folder" ; \
|
||||||
rm -rf "$$folder"; \
|
rm -rf "$$folder"; \
|
||||||
@@ -45,11 +73,6 @@ setup: ## Install dependencies for tests and coverage
|
|||||||
pip install gcovr; \
|
pip install gcovr; \
|
||||||
fi
|
fi
|
||||||
|
|
||||||
dest ?= ${HOME}/bin
|
|
||||||
main: ## Build only the b_main target
|
|
||||||
@cmake --build $(f_release) -t b_main --parallel
|
|
||||||
@cp $(f_release)/src/b_main $(dest)
|
|
||||||
|
|
||||||
dest ?= ${HOME}/bin
|
dest ?= ${HOME}/bin
|
||||||
install: ## Copy binary files to bin folder
|
install: ## Copy binary files to bin folder
|
||||||
@echo "Destination folder: $(dest)"
|
@echo "Destination folder: $(dest)"
|
||||||
@@ -70,34 +93,27 @@ dependency: ## Create a dependency graph diagram of the project (build/dependenc
|
|||||||
cd $(f_debug) && cmake .. --graphviz=dependency.dot && dot -Tpng dependency.dot -o dependency.png
|
cd $(f_debug) && cmake .. --graphviz=dependency.dot && dot -Tpng dependency.dot -o dependency.png
|
||||||
|
|
||||||
buildd: ## Build the debug targets
|
buildd: ## Build the debug targets
|
||||||
@cmake --build $(f_debug) -t $(app_targets) PlatformSample --parallel
|
@$(call compile_target,"Debug","$(f_debug)")
|
||||||
|
|
||||||
buildr: ## Build the release targets
|
buildr: ## Build the release targets
|
||||||
@cmake --build $(f_release) -t $(app_targets) --parallel
|
@$(call compile_target,"Release","$(f_release)")
|
||||||
|
|
||||||
clang-uml: ## Create uml class and sequence diagrams
|
clang-uml: ## Create uml class and sequence diagrams
|
||||||
clang-uml -p --add-compile-flag -I /usr/lib/gcc/x86_64-redhat-linux/8/include/
|
clang-uml -p --add-compile-flag -I /usr/lib/gcc/x86_64-redhat-linux/8/include/
|
||||||
|
|
||||||
debug: ## Build a debug version of the project with BayesNet from vcpkg
|
debug: ## Build a debug version of the project with Conan
|
||||||
@echo ">>> Building Debug Platform...";
|
@$(call build_target,"Debug","$(f_debug)", "ENABLE_TESTING=ON")
|
||||||
@if [ -d ./$(f_debug) ]; then rm -rf ./$(f_debug); fi
|
|
||||||
@mkdir $(f_debug);
|
release: ## Build a Release version of the project with Conan
|
||||||
@cmake -S . -B $(f_debug) -D CMAKE_BUILD_TYPE=Debug -D ENABLE_TESTING=ON -D CODE_COVERAGE=ON -D CMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE=${VCPKG_ROOT}/scripts/buildsystems/vcpkg.cmake
|
@$(call build_target,"Release","$(f_release)", "ENABLE_TESTING=OFF")
|
||||||
@echo ">>> Done";
|
|
||||||
|
|
||||||
release: ## Build a Release version of the project with BayesNet from vcpkg
|
|
||||||
@echo ">>> Building Release Platform...";
|
|
||||||
@if [ -d ./$(f_release) ]; then rm -rf ./$(f_release); fi
|
|
||||||
@mkdir $(f_release);
|
|
||||||
@cmake -S . -B $(f_release) -D CMAKE_BUILD_TYPE=Release -D CMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE=${VCPKG_ROOT}/scripts/buildsystems/vcpkg.cmake
|
|
||||||
@echo ">>> Done";
|
|
||||||
|
|
||||||
opt = ""
|
opt = ""
|
||||||
test: ## Run tests (opt="-s") to verbose output the tests, (opt="-c='Test Maximum Spanning Tree'") to run only that section
|
test: ## Run tests (opt="-s") to verbose output the tests, (opt="-c='Test Maximum Spanning Tree'") to run only that section
|
||||||
@echo ">>> Running Platform tests...";
|
@echo ">>> Running Platform tests...";
|
||||||
@$(MAKE) clean
|
@$(MAKE) clean
|
||||||
@$(MAKE) debug
|
@$(MAKE) debug
|
||||||
@cmake --build $(f_debug) -t $(test_targets) --parallel
|
@$(call "Compile_target", "Debug", "$(f_debug)", $(test_targets))
|
||||||
@for t in $(test_targets); do \
|
@for t in $(test_targets); do \
|
||||||
if [ -f $(f_debug)/tests/$$t ]; then \
|
if [ -f $(f_debug)/tests/$$t ]; then \
|
||||||
cd $(f_debug)/tests ; \
|
cd $(f_debug)/tests ; \
|
||||||
|
42
conanfile.py
Normal file
42
conanfile.py
Normal file
@@ -0,0 +1,42 @@
|
|||||||
|
from conan import ConanFile
|
||||||
|
from conan.tools.cmake import CMakeToolchain, CMakeDeps, cmake_layout
|
||||||
|
|
||||||
|
|
||||||
|
class PlatformConan(ConanFile):
|
||||||
|
name = "platform"
|
||||||
|
version = "1.1.0"
|
||||||
|
|
||||||
|
# Binary configuration
|
||||||
|
settings = "os", "compiler", "build_type", "arch"
|
||||||
|
|
||||||
|
# Sources are located in the same place as this recipe, copy them to the recipe
|
||||||
|
exports_sources = "CMakeLists.txt", "src/*", "tests/*", "config/*", "cmake/*"
|
||||||
|
|
||||||
|
def requirements(self):
|
||||||
|
# Core dependencies from vcpkg.json
|
||||||
|
self.requires("argparse/3.2")
|
||||||
|
self.requires("libtorch/2.7.1")
|
||||||
|
self.requires("nlohmann_json/3.11.3")
|
||||||
|
self.requires("folding/1.1.2")
|
||||||
|
self.requires("fimdlp/2.1.1")
|
||||||
|
self.requires("arff-files/1.2.1")
|
||||||
|
self.requires("bayesnet/1.2.1")
|
||||||
|
self.requires("pyclassifiers/1.0.3")
|
||||||
|
self.requires("libxlsxwriter/1.2.2")
|
||||||
|
|
||||||
|
def build_requirements(self):
|
||||||
|
self.tool_requires("cmake/[>=3.30]")
|
||||||
|
self.test_requires("catch2/3.8.1")
|
||||||
|
|
||||||
|
def layout(self):
|
||||||
|
cmake_layout(self)
|
||||||
|
|
||||||
|
def generate(self):
|
||||||
|
deps = CMakeDeps(self)
|
||||||
|
deps.generate()
|
||||||
|
tc = CMakeToolchain(self)
|
||||||
|
tc.generate()
|
||||||
|
|
||||||
|
def configure(self):
|
||||||
|
# C++20 requirement
|
||||||
|
self.settings.compiler.cppstd = "20"
|
@@ -1,13 +1,8 @@
|
|||||||
include_directories(
|
include_directories(
|
||||||
## Libs
|
|
||||||
${Python3_INCLUDE_DIRS}
|
${Python3_INCLUDE_DIRS}
|
||||||
${MPI_CXX_INCLUDE_DIRS}
|
${MPI_CXX_INCLUDE_DIRS}
|
||||||
${TORCH_INCLUDE_DIRS}
|
|
||||||
${CMAKE_BINARY_DIR}/configured_files/include
|
${CMAKE_BINARY_DIR}/configured_files/include
|
||||||
${PyClassifiers_INCLUDE_DIRS}
|
|
||||||
## Platform
|
|
||||||
${Platform_SOURCE_DIR}/src
|
${Platform_SOURCE_DIR}/src
|
||||||
${Platform_SOURCE_DIR}/results
|
|
||||||
)
|
)
|
||||||
|
|
||||||
# b_best
|
# b_best
|
||||||
@@ -23,7 +18,7 @@ add_executable(
|
|||||||
experimental_clfs/DecisionTree.cpp
|
experimental_clfs/DecisionTree.cpp
|
||||||
experimental_clfs/AdaBoost.cpp
|
experimental_clfs/AdaBoost.cpp
|
||||||
)
|
)
|
||||||
target_link_libraries(b_best Boost::boost "${PyClassifiers}" bayesnet::bayesnet fimdlp ${Python3_LIBRARIES} "${TORCH_LIBRARIES}" Boost::python Boost::numpy "${XLSXWRITER_LIB}")
|
target_link_libraries(b_best Boost::boost pyclassifiers::pyclassifiers bayesnet::bayesnet argparse::argparse fimdlp::fimdlp ${Python3_LIBRARIES} torch::torch Boost::python Boost::numpy libxlsxwriter::libxlsxwriter)
|
||||||
|
|
||||||
# b_grid
|
# b_grid
|
||||||
set(grid_sources GridSearch.cpp GridData.cpp GridExperiment.cpp GridBase.cpp )
|
set(grid_sources GridSearch.cpp GridData.cpp GridExperiment.cpp GridBase.cpp )
|
||||||
@@ -38,7 +33,7 @@ add_executable(b_grid commands/b_grid.cpp ${grid_sources}
|
|||||||
experimental_clfs/DecisionTree.cpp
|
experimental_clfs/DecisionTree.cpp
|
||||||
experimental_clfs/AdaBoost.cpp
|
experimental_clfs/AdaBoost.cpp
|
||||||
)
|
)
|
||||||
target_link_libraries(b_grid ${MPI_CXX_LIBRARIES} "${PyClassifiers}" bayesnet::bayesnet fimdlp ${Python3_LIBRARIES} "${TORCH_LIBRARIES}" Boost::python Boost::numpy)
|
target_link_libraries(b_grid ${MPI_CXX_LIBRARIES} pyclassifiers::pyclassifiers bayesnet::bayesnet argparse::argparse fimdlp::fimdlp ${Python3_LIBRARIES} torch::torch Boost::python Boost::numpy)
|
||||||
|
|
||||||
# b_list
|
# b_list
|
||||||
add_executable(b_list commands/b_list.cpp
|
add_executable(b_list commands/b_list.cpp
|
||||||
@@ -51,7 +46,7 @@ add_executable(b_list commands/b_list.cpp
|
|||||||
experimental_clfs/DecisionTree.cpp
|
experimental_clfs/DecisionTree.cpp
|
||||||
experimental_clfs/AdaBoost.cpp
|
experimental_clfs/AdaBoost.cpp
|
||||||
)
|
)
|
||||||
target_link_libraries(b_list "${PyClassifiers}" bayesnet::bayesnet fimdlp ${Python3_LIBRARIES} "${TORCH_LIBRARIES}" Boost::python Boost::numpy "${XLSXWRITER_LIB}")
|
target_link_libraries(b_list pyclassifiers::pyclassifiers bayesnet::bayesnet argparse::argparse fimdlp::fimdlp ${Python3_LIBRARIES} torch::torch Boost::python Boost::numpy libxlsxwriter::libxlsxwriter)
|
||||||
|
|
||||||
# b_main
|
# b_main
|
||||||
set(main_sources Experiment.cpp Models.cpp HyperParameters.cpp Scores.cpp ArgumentsExperiment.cpp)
|
set(main_sources Experiment.cpp Models.cpp HyperParameters.cpp Scores.cpp ArgumentsExperiment.cpp)
|
||||||
@@ -66,7 +61,7 @@ add_executable(b_main commands/b_main.cpp ${main_sources}
|
|||||||
experimental_clfs/DecisionTree.cpp
|
experimental_clfs/DecisionTree.cpp
|
||||||
experimental_clfs/AdaBoost.cpp
|
experimental_clfs/AdaBoost.cpp
|
||||||
)
|
)
|
||||||
target_link_libraries(b_main PRIVATE nlohmann_json::nlohmann_json "${PyClassifiers}" bayesnet::bayesnet fimdlp ${Python3_LIBRARIES} "${TORCH_LIBRARIES}" Boost::python Boost::numpy)
|
target_link_libraries(b_main PRIVATE nlohmann_json::nlohmann_json pyclassifiers::pyclassifiers bayesnet::bayesnet argparse::argparse fimdlp::fimdlp ${Python3_LIBRARIES} torch::torch Boost::python Boost::numpy)
|
||||||
|
|
||||||
# b_manage
|
# b_manage
|
||||||
set(manage_sources ManageScreen.cpp OptionsMenu.cpp ResultsManager.cpp)
|
set(manage_sources ManageScreen.cpp OptionsMenu.cpp ResultsManager.cpp)
|
||||||
@@ -78,7 +73,8 @@ add_executable(
|
|||||||
results/Result.cpp results/ResultsDataset.cpp results/ResultsDatasetConsole.cpp
|
results/Result.cpp results/ResultsDataset.cpp results/ResultsDatasetConsole.cpp
|
||||||
main/Scores.cpp
|
main/Scores.cpp
|
||||||
)
|
)
|
||||||
target_link_libraries(b_manage "${TORCH_LIBRARIES}" "${XLSXWRITER_LIB}" fimdlp bayesnet::bayesnet)
|
target_link_libraries(b_manage torch::torch libxlsxwriter::libxlsxwriter fimdlp::fimdlp bayesnet::bayesnet argparse::argparse)
|
||||||
|
|
||||||
# b_results
|
# b_results
|
||||||
add_executable(b_results commands/b_results.cpp)
|
add_executable(b_results commands/b_results.cpp)
|
||||||
|
target_link_libraries(b_results torch::torch libxlsxwriter::libxlsxwriter fimdlp::fimdlp bayesnet::bayesnet argparse::argparse)
|
||||||
|
@@ -2,8 +2,8 @@
|
|||||||
#include <filesystem>
|
#include <filesystem>
|
||||||
#include <fstream>
|
#include <fstream>
|
||||||
#include <vector>
|
#include <vector>
|
||||||
#include <argparse/argparse.hpp>
|
#include "nlohmann/json.hpp"
|
||||||
#include <nlohmann/json.hpp>
|
#include "argparse/argparse.hpp"
|
||||||
#include "common/Paths.h"
|
#include "common/Paths.h"
|
||||||
#include "results/JsonValidator.h"
|
#include "results/JsonValidator.h"
|
||||||
#include "results/SchemaV1_0.h"
|
#include "results/SchemaV1_0.h"
|
||||||
|
@@ -1,4 +1,4 @@
|
|||||||
#include <ArffFiles/ArffFiles.hpp>
|
#include <ArffFiles.hpp>
|
||||||
#include <fstream>
|
#include <fstream>
|
||||||
#include "Dataset.h"
|
#include "Dataset.h"
|
||||||
namespace platform {
|
namespace platform {
|
||||||
|
@@ -5,6 +5,15 @@
|
|||||||
namespace platform {
|
namespace platform {
|
||||||
class TensorUtils {
|
class TensorUtils {
|
||||||
public:
|
public:
|
||||||
|
template <typename T>
|
||||||
|
static std::vector<T> tensorToVector(const torch::Tensor& tensor)
|
||||||
|
{
|
||||||
|
torch::Tensor contig_tensor = tensor.contiguous();
|
||||||
|
auto num_elements = contig_tensor.numel();
|
||||||
|
const T* tensor_data = contig_tensor.data_ptr<T>();
|
||||||
|
std::vector<T> result(tensor_data, tensor_data + num_elements);
|
||||||
|
return result;
|
||||||
|
}
|
||||||
static std::vector<std::vector<int>> to_matrix(const torch::Tensor& X)
|
static std::vector<std::vector<int>> to_matrix(const torch::Tensor& X)
|
||||||
{
|
{
|
||||||
// Ensure tensor is contiguous in memory
|
// Ensure tensor is contiguous in memory
|
||||||
@@ -53,7 +62,7 @@ namespace platform {
|
|||||||
torch::Tensor tensor = torch::empty({ static_cast<long>(rows), static_cast<long>(cols) }, torch::kInt64);
|
torch::Tensor tensor = torch::empty({ static_cast<long>(rows), static_cast<long>(cols) }, torch::kInt64);
|
||||||
for (size_t i = 0; i < rows; ++i) {
|
for (size_t i = 0; i < rows; ++i) {
|
||||||
for (size_t j = 0; j < cols; ++j) {
|
for (size_t j = 0; j < cols; ++j) {
|
||||||
tensor.index_put_({ static_cast<long>(i), static_cast<long>(j) }, data[i][j]);
|
tensor.index_put_({static_cast<int64_t>(i), static_cast<int64_t>(j)}, torch::scalar_tensor(data[i][j]));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return tensor;
|
return tensor;
|
@@ -6,17 +6,15 @@
|
|||||||
#include <string>
|
#include <string>
|
||||||
#include <vector>
|
#include <vector>
|
||||||
#include <algorithm>
|
#include <algorithm>
|
||||||
#include <torch/torch.h>
|
#include <cstdlib>
|
||||||
|
#include <cmath>
|
||||||
|
#include <ctime>
|
||||||
|
#include <iomanip>
|
||||||
|
#include <string.h>
|
||||||
|
|
||||||
|
extern char** environ;
|
||||||
|
|
||||||
namespace platform {
|
namespace platform {
|
||||||
template <typename T>
|
|
||||||
std::vector<T> tensorToVector(const torch::Tensor& tensor)
|
|
||||||
{
|
|
||||||
torch::Tensor contig_tensor = tensor.contiguous();
|
|
||||||
auto num_elements = contig_tensor.numel();
|
|
||||||
const T* tensor_data = contig_tensor.data_ptr<T>();
|
|
||||||
std::vector<T> result(tensor_data, tensor_data + num_elements);
|
|
||||||
return result;
|
|
||||||
}
|
|
||||||
static std::string trim(const std::string& str)
|
static std::string trim(const std::string& str)
|
||||||
{
|
{
|
||||||
std::string result = str;
|
std::string result = str;
|
||||||
|
@@ -11,7 +11,15 @@
|
|||||||
#include <numeric>
|
#include <numeric>
|
||||||
#include <sstream>
|
#include <sstream>
|
||||||
#include <iomanip>
|
#include <iomanip>
|
||||||
#include "TensorUtils.hpp"
|
#include "common/TensorUtils.hpp"
|
||||||
|
|
||||||
|
// Conditional debug macro for performance-critical sections
|
||||||
|
#define DEBUG_LOG(condition, ...) \
|
||||||
|
do { \
|
||||||
|
if (__builtin_expect((condition), 0)) { \
|
||||||
|
std::cout << __VA_ARGS__ << std::endl; \
|
||||||
|
} \
|
||||||
|
} while(0)
|
||||||
|
|
||||||
namespace bayesnet {
|
namespace bayesnet {
|
||||||
|
|
||||||
@@ -21,6 +29,8 @@ namespace bayesnet {
|
|||||||
validHyperparameters = { "n_estimators", "base_max_depth" };
|
validHyperparameters = { "n_estimators", "base_max_depth" };
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Versión optimizada de buildModel - Reemplazar en AdaBoost.cpp:
|
||||||
|
|
||||||
void AdaBoost::buildModel(const torch::Tensor& weights)
|
void AdaBoost::buildModel(const torch::Tensor& weights)
|
||||||
{
|
{
|
||||||
// Initialize variables
|
// Initialize variables
|
||||||
@@ -38,20 +48,23 @@ namespace bayesnet {
|
|||||||
|
|
||||||
// If initial weights are provided, incorporate them
|
// If initial weights are provided, incorporate them
|
||||||
if (weights.defined() && weights.numel() > 0) {
|
if (weights.defined() && weights.numel() > 0) {
|
||||||
sample_weights *= weights;
|
if (weights.size(0) != n_samples) {
|
||||||
|
throw std::runtime_error("weights must have the same length as number of samples");
|
||||||
|
}
|
||||||
|
sample_weights = weights.clone();
|
||||||
normalizeWeights();
|
normalizeWeights();
|
||||||
}
|
}
|
||||||
|
|
||||||
// Debug information
|
// Conditional debug information (only when debug is enabled)
|
||||||
if (debug) {
|
DEBUG_LOG(debug, "Starting AdaBoost training with " << n_estimators << " estimators\n"
|
||||||
std::cout << "Starting AdaBoost training with " << n_estimators << " estimators" << std::endl;
|
<< "Number of classes: " << n_classes << "\n"
|
||||||
std::cout << "Number of classes: " << n_classes << std::endl;
|
<< "Number of features: " << n << "\n"
|
||||||
std::cout << "Number of features: " << n << std::endl;
|
<< "Number of samples: " << n_samples);
|
||||||
std::cout << "Number of samples: " << n_samples << std::endl;
|
|
||||||
}
|
// Pre-compute random guess error threshold
|
||||||
|
const double random_guess_error = 1.0 - (1.0 / static_cast<double>(n_classes));
|
||||||
|
|
||||||
// Main AdaBoost training loop (SAMME algorithm)
|
// Main AdaBoost training loop (SAMME algorithm)
|
||||||
// (Stagewise Additive Modeling using a Multi - class Exponential loss)
|
|
||||||
for (int iter = 0; iter < n_estimators; ++iter) {
|
for (int iter = 0; iter < n_estimators; ++iter) {
|
||||||
// Train base estimator with current sample weights
|
// Train base estimator with current sample weights
|
||||||
auto estimator = trainBaseEstimator(sample_weights);
|
auto estimator = trainBaseEstimator(sample_weights);
|
||||||
@@ -60,12 +73,9 @@ namespace bayesnet {
|
|||||||
double weighted_error = calculateWeightedError(estimator.get(), sample_weights);
|
double weighted_error = calculateWeightedError(estimator.get(), sample_weights);
|
||||||
training_errors.push_back(weighted_error);
|
training_errors.push_back(weighted_error);
|
||||||
|
|
||||||
// Check if error is too high (worse than random guessing)
|
|
||||||
double random_guess_error = 1.0 - (1.0 / n_classes);
|
|
||||||
|
|
||||||
// According to SAMME, we need error < random_guess_error
|
// According to SAMME, we need error < random_guess_error
|
||||||
if (weighted_error >= random_guess_error) {
|
if (weighted_error >= random_guess_error) {
|
||||||
if (debug) std::cout << " Error >= random guess (" << random_guess_error << "), stopping" << std::endl;
|
DEBUG_LOG(debug, "Error >= random guess (" << random_guess_error << "), stopping");
|
||||||
// If only one estimator and it's worse than random, keep it with zero weight
|
// If only one estimator and it's worse than random, keep it with zero weight
|
||||||
if (models.empty()) {
|
if (models.empty()) {
|
||||||
models.push_back(std::move(estimator));
|
models.push_back(std::move(estimator));
|
||||||
@@ -76,7 +86,7 @@ namespace bayesnet {
|
|||||||
|
|
||||||
// Check for perfect classification BEFORE calculating alpha
|
// Check for perfect classification BEFORE calculating alpha
|
||||||
if (weighted_error <= 1e-10) {
|
if (weighted_error <= 1e-10) {
|
||||||
if (debug) std::cout << " Perfect classification achieved (error=" << weighted_error << ")" << std::endl;
|
DEBUG_LOG(debug, "Perfect classification achieved (error=" << weighted_error << ")");
|
||||||
|
|
||||||
// For perfect classification, use a large but finite alpha
|
// For perfect classification, use a large but finite alpha
|
||||||
double alpha = 10.0 + std::log(static_cast<double>(n_classes - 1));
|
double alpha = 10.0 + std::log(static_cast<double>(n_classes - 1));
|
||||||
@@ -85,12 +95,10 @@ namespace bayesnet {
|
|||||||
models.push_back(std::move(estimator));
|
models.push_back(std::move(estimator));
|
||||||
alphas.push_back(alpha);
|
alphas.push_back(alpha);
|
||||||
|
|
||||||
if (debug) {
|
DEBUG_LOG(debug, "Iteration " << iter << ":\n"
|
||||||
std::cout << "Iteration " << iter << ":" << std::endl;
|
<< " Weighted error: " << weighted_error << "\n"
|
||||||
std::cout << " Weighted error: " << weighted_error << std::endl;
|
<< " Alpha (finite): " << alpha << "\n"
|
||||||
std::cout << " Alpha (finite): " << alpha << std::endl;
|
<< " Random guess error: " << random_guess_error);
|
||||||
std::cout << " Random guess error: " << random_guess_error << std::endl;
|
|
||||||
}
|
|
||||||
|
|
||||||
break; // Stop training as we have a perfect classifier
|
break; // Stop training as we have a perfect classifier
|
||||||
}
|
}
|
||||||
@@ -115,18 +123,15 @@ namespace bayesnet {
|
|||||||
normalizeWeights();
|
normalizeWeights();
|
||||||
}
|
}
|
||||||
|
|
||||||
if (debug) {
|
DEBUG_LOG(debug, "Iteration " << iter << ":\n"
|
||||||
std::cout << "Iteration " << iter << ":" << std::endl;
|
<< " Weighted error: " << weighted_error << "\n"
|
||||||
std::cout << " Weighted error: " << weighted_error << std::endl;
|
<< " Alpha: " << alpha << "\n"
|
||||||
std::cout << " Alpha: " << alpha << std::endl;
|
<< " Random guess error: " << random_guess_error);
|
||||||
std::cout << " Random guess error: " << random_guess_error << std::endl;
|
|
||||||
std::cout << " Random guess error: " << random_guess_error << std::endl;
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// Set the number of models actually trained
|
// Set the number of models actually trained
|
||||||
n_models = models.size();
|
n_models = models.size();
|
||||||
if (debug) std::cout << "AdaBoost training completed with " << n_models << " models" << std::endl;
|
DEBUG_LOG(debug, "AdaBoost training completed with " << n_models << " models");
|
||||||
}
|
}
|
||||||
|
|
||||||
void AdaBoost::trainModel(const torch::Tensor& weights, const Smoothing_t smoothing)
|
void AdaBoost::trainModel(const torch::Tensor& weights, const Smoothing_t smoothing)
|
||||||
@@ -152,44 +157,60 @@ namespace bayesnet {
|
|||||||
|
|
||||||
double AdaBoost::calculateWeightedError(Classifier* estimator, const torch::Tensor& weights)
|
double AdaBoost::calculateWeightedError(Classifier* estimator, const torch::Tensor& weights)
|
||||||
{
|
{
|
||||||
// Get features and labels from dataset
|
// Get features and labels from dataset (avoid repeated indexing)
|
||||||
auto X = dataset.index({ torch::indexing::Slice(0, dataset.size(0) - 1), torch::indexing::Slice() });
|
auto X = dataset.index({ torch::indexing::Slice(0, dataset.size(0) - 1), torch::indexing::Slice() });
|
||||||
auto y_true = dataset.index({ -1, torch::indexing::Slice() });
|
auto y_true = dataset.index({ -1, torch::indexing::Slice() });
|
||||||
|
|
||||||
// Get predictions from the estimator
|
// Get predictions from the estimator
|
||||||
auto y_pred = estimator->predict(X);
|
auto y_pred = estimator->predict(X);
|
||||||
|
|
||||||
// Calculate weighted error
|
// Vectorized error calculation using PyTorch operations
|
||||||
auto incorrect = (y_pred != y_true).to(torch::kFloat);
|
auto incorrect = (y_pred != y_true).to(torch::kDouble);
|
||||||
|
|
||||||
// Ensure weights are normalized
|
// Direct dot product for weighted error (more efficient than sum)
|
||||||
auto normalized_weights = weights / weights.sum();
|
double weighted_error = torch::dot(incorrect, weights).item<double>();
|
||||||
|
|
||||||
// Calculate weighted error
|
// Clamp to valid range in one operation
|
||||||
double weighted_error = torch::sum(incorrect * normalized_weights).item<double>();
|
return std::clamp(weighted_error, 1e-15, 1.0 - 1e-15);
|
||||||
|
|
||||||
return weighted_error;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
void AdaBoost::updateSampleWeights(Classifier* estimator, double alpha)
|
void AdaBoost::updateSampleWeights(Classifier* estimator, double alpha)
|
||||||
{
|
{
|
||||||
// Get predictions from the estimator
|
// Get predictions from the estimator (reuse from calculateWeightedError if possible)
|
||||||
auto X = dataset.index({ torch::indexing::Slice(0, dataset.size(0) - 1), torch::indexing::Slice() });
|
auto X = dataset.index({ torch::indexing::Slice(0, dataset.size(0) - 1), torch::indexing::Slice() });
|
||||||
auto y_true = dataset.index({ -1, torch::indexing::Slice() });
|
auto y_true = dataset.index({ -1, torch::indexing::Slice() });
|
||||||
auto y_pred = estimator->predict(X);
|
auto y_pred = estimator->predict(X);
|
||||||
|
|
||||||
// Update weights according to SAMME algorithm
|
// Vectorized weight update using PyTorch operations
|
||||||
// w_i = w_i * exp(alpha * I(y_i != y_pred_i))
|
auto incorrect = (y_pred != y_true).to(torch::kDouble);
|
||||||
auto incorrect = (y_pred != y_true).to(torch::kFloat);
|
|
||||||
|
// Single vectorized operation instead of element-wise multiplication
|
||||||
sample_weights *= torch::exp(alpha * incorrect);
|
sample_weights *= torch::exp(alpha * incorrect);
|
||||||
|
|
||||||
|
// Vectorized clamping for numerical stability
|
||||||
|
sample_weights = torch::clamp(sample_weights, 1e-15, 1e15);
|
||||||
}
|
}
|
||||||
|
|
||||||
void AdaBoost::normalizeWeights()
|
void AdaBoost::normalizeWeights()
|
||||||
{
|
{
|
||||||
// Normalize weights to sum to 1
|
// Single-pass normalization using PyTorch operations
|
||||||
double sum_weights = torch::sum(sample_weights).item<double>();
|
double sum_weights = torch::sum(sample_weights).item<double>();
|
||||||
if (sum_weights > 0) {
|
|
||||||
|
if (__builtin_expect(sum_weights <= 0, 0)) {
|
||||||
|
// Reset to uniform if all weights are zero/negative (rare case)
|
||||||
|
sample_weights = torch::ones_like(sample_weights) / sample_weights.size(0);
|
||||||
|
} else {
|
||||||
|
// Vectorized normalization
|
||||||
sample_weights /= sum_weights;
|
sample_weights /= sum_weights;
|
||||||
|
|
||||||
|
// Vectorized minimum weight enforcement
|
||||||
|
sample_weights = torch::clamp_min(sample_weights, 1e-15);
|
||||||
|
|
||||||
|
// Renormalize after clamping (if any weights were clamped)
|
||||||
|
double new_sum = torch::sum(sample_weights).item<double>();
|
||||||
|
if (new_sum != 1.0) {
|
||||||
|
sample_weights /= new_sum;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -273,194 +294,199 @@ namespace bayesnet {
|
|||||||
Ensemble::setHyperparameters(hyperparameters);
|
Ensemble::setHyperparameters(hyperparameters);
|
||||||
}
|
}
|
||||||
|
|
||||||
torch::Tensor AdaBoost::predict(torch::Tensor& X)
|
int AdaBoost::predictSample(const torch::Tensor& x) const
|
||||||
{
|
{
|
||||||
if (!fitted) {
|
// Early validation (keep essential checks only)
|
||||||
|
if (!fitted || models.empty()) {
|
||||||
throw std::runtime_error(CLASSIFIER_NOT_FITTED);
|
throw std::runtime_error(CLASSIFIER_NOT_FITTED);
|
||||||
}
|
}
|
||||||
|
|
||||||
if (models.empty()) {
|
// Pre-allocate and reuse memory
|
||||||
throw std::runtime_error("No models have been trained");
|
static thread_local std::vector<double> class_votes_cache;
|
||||||
|
if (class_votes_cache.size() != static_cast<size_t>(n_classes)) {
|
||||||
|
class_votes_cache.resize(n_classes);
|
||||||
|
}
|
||||||
|
std::fill(class_votes_cache.begin(), class_votes_cache.end(), 0.0);
|
||||||
|
|
||||||
|
// Optimized voting loop - avoid exception handling in hot path
|
||||||
|
for (size_t i = 0; i < models.size(); ++i) {
|
||||||
|
double alpha = alphas[i];
|
||||||
|
if (alpha <= 0 || !std::isfinite(alpha)) continue;
|
||||||
|
|
||||||
|
// Direct cast and call - avoid virtual dispatch overhead
|
||||||
|
int predicted_class = static_cast<DecisionTree*>(models[i].get())->predictSample(x);
|
||||||
|
|
||||||
|
// Bounds check with branch prediction hint
|
||||||
|
if (__builtin_expect(predicted_class >= 0 && predicted_class < n_classes, 1)) {
|
||||||
|
class_votes_cache[predicted_class] += alpha;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// X should be (n_features, n_samples)
|
// Fast argmax using iterators
|
||||||
if (X.size(0) != n) {
|
return std::distance(class_votes_cache.begin(),
|
||||||
throw std::runtime_error("Input has wrong number of features. Expected " +
|
std::max_element(class_votes_cache.begin(), class_votes_cache.end()));
|
||||||
std::to_string(n) + " but got " + std::to_string(X.size(0)));
|
|
||||||
}
|
}
|
||||||
|
|
||||||
int n_samples = X.size(1);
|
torch::Tensor AdaBoost::predictProbaSample(const torch::Tensor& x) const
|
||||||
torch::Tensor predictions = torch::zeros({ n_samples }, torch::kInt32);
|
{
|
||||||
|
// Early validation
|
||||||
for (int i = 0; i < n_samples; i++) {
|
if (!fitted || models.empty()) {
|
||||||
auto sample = X.index({ torch::indexing::Slice(), i });
|
throw std::runtime_error(CLASSIFIER_NOT_FITTED);
|
||||||
predictions[i] = predictSample(sample);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
return predictions;
|
// Use stack allocation for small arrays (typical case: n_classes <= 32)
|
||||||
|
constexpr int STACK_THRESHOLD = 32;
|
||||||
|
double stack_votes[STACK_THRESHOLD];
|
||||||
|
std::vector<double> heap_votes;
|
||||||
|
double* class_votes;
|
||||||
|
|
||||||
|
if (n_classes <= STACK_THRESHOLD) {
|
||||||
|
class_votes = stack_votes;
|
||||||
|
std::fill_n(class_votes, n_classes, 0.0);
|
||||||
|
} else {
|
||||||
|
heap_votes.resize(n_classes, 0.0);
|
||||||
|
class_votes = heap_votes.data();
|
||||||
|
}
|
||||||
|
|
||||||
|
double total_votes = 0.0;
|
||||||
|
|
||||||
|
// Optimized voting loop
|
||||||
|
for (size_t i = 0; i < models.size(); ++i) {
|
||||||
|
double alpha = alphas[i];
|
||||||
|
if (alpha <= 0 || !std::isfinite(alpha)) continue;
|
||||||
|
|
||||||
|
int predicted_class = static_cast<DecisionTree*>(models[i].get())->predictSample(x);
|
||||||
|
|
||||||
|
if (__builtin_expect(predicted_class >= 0 && predicted_class < n_classes, 1)) {
|
||||||
|
class_votes[predicted_class] += alpha;
|
||||||
|
total_votes += alpha;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Direct tensor creation with pre-computed size
|
||||||
|
torch::Tensor class_probs = torch::empty({ n_classes }, torch::TensorOptions().dtype(torch::kFloat32));
|
||||||
|
auto probs_accessor = class_probs.accessor<float, 1>();
|
||||||
|
|
||||||
|
if (__builtin_expect(total_votes > 0.0, 1)) {
|
||||||
|
// Vectorized probability calculation
|
||||||
|
const double inv_total = 1.0 / total_votes;
|
||||||
|
for (int j = 0; j < n_classes; ++j) {
|
||||||
|
probs_accessor[j] = static_cast<float>(class_votes[j] * inv_total);
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
// Uniform distribution fallback
|
||||||
|
const float uniform_prob = 1.0f / n_classes;
|
||||||
|
for (int j = 0; j < n_classes; ++j) {
|
||||||
|
probs_accessor[j] = uniform_prob;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return class_probs;
|
||||||
}
|
}
|
||||||
|
|
||||||
torch::Tensor AdaBoost::predict_proba(torch::Tensor& X)
|
torch::Tensor AdaBoost::predict_proba(torch::Tensor& X)
|
||||||
{
|
{
|
||||||
if (!fitted) {
|
if (!fitted || models.empty()) {
|
||||||
throw std::runtime_error(CLASSIFIER_NOT_FITTED);
|
throw std::runtime_error(CLASSIFIER_NOT_FITTED);
|
||||||
}
|
}
|
||||||
|
|
||||||
if (models.empty()) {
|
// Input validation
|
||||||
throw std::runtime_error("No models have been trained");
|
|
||||||
}
|
|
||||||
|
|
||||||
// X should be (n_features, n_samples)
|
|
||||||
if (X.size(0) != n) {
|
if (X.size(0) != n) {
|
||||||
throw std::runtime_error("Input has wrong number of features. Expected " +
|
throw std::runtime_error("Input has wrong number of features. Expected " +
|
||||||
std::to_string(n) + " but got " + std::to_string(X.size(0)));
|
std::to_string(n) + " but got " + std::to_string(X.size(0)));
|
||||||
}
|
}
|
||||||
|
|
||||||
int n_samples = X.size(1);
|
const int n_samples = X.size(1);
|
||||||
torch::Tensor probabilities = torch::zeros({ n_samples, n_classes });
|
|
||||||
|
|
||||||
for (int i = 0; i < n_samples; i++) {
|
// Pre-allocate output tensor with correct layout
|
||||||
auto sample = X.index({ torch::indexing::Slice(), i });
|
torch::Tensor probabilities = torch::empty({ n_samples, n_classes },
|
||||||
|
torch::TensorOptions().dtype(torch::kFloat32));
|
||||||
|
|
||||||
|
// Convert to contiguous memory if needed (optimization for memory access)
|
||||||
|
if (!X.is_contiguous()) {
|
||||||
|
X = X.contiguous();
|
||||||
|
}
|
||||||
|
|
||||||
|
// Batch processing with memory-efficient sample extraction
|
||||||
|
for (int i = 0; i < n_samples; ++i) {
|
||||||
|
// Extract sample without unnecessary copies
|
||||||
|
auto sample = X.select(1, i);
|
||||||
|
|
||||||
|
// Direct assignment to pre-allocated tensor
|
||||||
probabilities[i] = predictProbaSample(sample);
|
probabilities[i] = predictProbaSample(sample);
|
||||||
}
|
}
|
||||||
|
|
||||||
return probabilities;
|
return probabilities;
|
||||||
}
|
}
|
||||||
|
|
||||||
std::vector<int> AdaBoost::predict(std::vector<std::vector<int>>& X)
|
|
||||||
{
|
|
||||||
// Convert to tensor - X is samples x features, need to transpose
|
|
||||||
torch::Tensor X_tensor = platform::TensorUtils::to_matrix(X);
|
|
||||||
auto predictions = predict(X_tensor);
|
|
||||||
std::vector<int> result = platform::TensorUtils::to_vector<int>(predictions);
|
|
||||||
return result;
|
|
||||||
}
|
|
||||||
|
|
||||||
std::vector<std::vector<double>> AdaBoost::predict_proba(std::vector<std::vector<int>>& X)
|
std::vector<std::vector<double>> AdaBoost::predict_proba(std::vector<std::vector<int>>& X)
|
||||||
{
|
{
|
||||||
auto n_samples = X[0].size();
|
const size_t n_samples = X[0].size();
|
||||||
// Convert to tensor - X is samples x features, need to transpose
|
|
||||||
|
// Pre-allocate result with exact size
|
||||||
|
std::vector<std::vector<double>> result;
|
||||||
|
result.reserve(n_samples);
|
||||||
|
|
||||||
|
// Avoid repeated allocations
|
||||||
|
for (size_t i = 0; i < n_samples; ++i) {
|
||||||
|
result.emplace_back(n_classes, 0.0);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Convert to tensor only once (batch conversion is more efficient)
|
||||||
torch::Tensor X_tensor = platform::TensorUtils::to_matrix(X);
|
torch::Tensor X_tensor = platform::TensorUtils::to_matrix(X);
|
||||||
auto proba_tensor = predict_proba(X_tensor);
|
torch::Tensor proba_tensor = predict_proba(X_tensor);
|
||||||
|
|
||||||
std::vector<std::vector<double>> result(n_samples, std::vector<double>(n_classes, 0.0));
|
// Optimized tensor-to-vector conversion
|
||||||
|
auto proba_accessor = proba_tensor.accessor<float, 2>();
|
||||||
for (size_t i = 0; i < n_samples; i++) {
|
for (size_t i = 0; i < n_samples; ++i) {
|
||||||
for (int j = 0; j < n_classes; j++) {
|
for (int j = 0; j < n_classes; ++j) {
|
||||||
result[i][j] = proba_tensor[i][j].item<double>();
|
result[i][j] = static_cast<double>(proba_accessor[i][j]);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
return result;
|
return result;
|
||||||
}
|
}
|
||||||
|
|
||||||
int AdaBoost::predictSample(const torch::Tensor& x) const
|
torch::Tensor AdaBoost::predict(torch::Tensor& X)
|
||||||
{
|
{
|
||||||
if (!fitted) {
|
if (!fitted || models.empty()) {
|
||||||
throw std::runtime_error(CLASSIFIER_NOT_FITTED);
|
throw std::runtime_error(CLASSIFIER_NOT_FITTED);
|
||||||
}
|
}
|
||||||
|
|
||||||
if (models.empty()) {
|
if (X.size(0) != n) {
|
||||||
throw std::runtime_error("No models have been trained");
|
throw std::runtime_error("Input has wrong number of features. Expected " +
|
||||||
|
std::to_string(n) + " but got " + std::to_string(X.size(0)));
|
||||||
}
|
}
|
||||||
|
|
||||||
// x should be a 1D tensor with n features
|
const int n_samples = X.size(1);
|
||||||
if (x.size(0) != n) {
|
|
||||||
throw std::runtime_error("Input sample has wrong number of features. Expected " +
|
// Pre-allocate with correct dtype
|
||||||
std::to_string(n) + " but got " + std::to_string(x.size(0)));
|
torch::Tensor predictions = torch::empty({ n_samples }, torch::TensorOptions().dtype(torch::kInt32));
|
||||||
|
auto pred_accessor = predictions.accessor<int32_t, 1>();
|
||||||
|
|
||||||
|
// Ensure contiguous memory layout
|
||||||
|
if (!X.is_contiguous()) {
|
||||||
|
X = X.contiguous();
|
||||||
}
|
}
|
||||||
|
|
||||||
// Initialize class votes
|
// Optimized prediction loop
|
||||||
std::vector<double> class_votes(n_classes, 0.0);
|
for (int i = 0; i < n_samples; ++i) {
|
||||||
|
auto sample = X.select(1, i);
|
||||||
// Accumulate weighted votes from all estimators
|
pred_accessor[i] = predictSample(sample);
|
||||||
for (size_t i = 0; i < models.size(); i++) {
|
|
||||||
if (alphas[i] <= 0) continue; // Skip estimators with zero or negative weight
|
|
||||||
try {
|
|
||||||
// Get prediction from this estimator
|
|
||||||
int predicted_class = static_cast<DecisionTree*>(models[i].get())->predictSample(x);
|
|
||||||
|
|
||||||
// Add weighted vote for this class
|
|
||||||
if (predicted_class >= 0 && predicted_class < n_classes) {
|
|
||||||
class_votes[predicted_class] += alphas[i];
|
|
||||||
}
|
|
||||||
}
|
|
||||||
catch (const std::exception& e) {
|
|
||||||
std::cerr << "Error in estimator " << i << ": " << e.what() << std::endl;
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// Return class with highest weighted vote
|
return predictions;
|
||||||
return std::distance(class_votes.begin(),
|
|
||||||
std::max_element(class_votes.begin(), class_votes.end()));
|
|
||||||
}
|
}
|
||||||
|
|
||||||
torch::Tensor AdaBoost::predictProbaSample(const torch::Tensor& x) const
|
std::vector<int> AdaBoost::predict(std::vector<std::vector<int>>& X)
|
||||||
{
|
{
|
||||||
if (!fitted) {
|
// Single tensor conversion for batch processing
|
||||||
throw std::runtime_error(CLASSIFIER_NOT_FITTED);
|
torch::Tensor X_tensor = platform::TensorUtils::to_matrix(X);
|
||||||
}
|
torch::Tensor predictions_tensor = predict(X_tensor);
|
||||||
|
|
||||||
if (models.empty()) {
|
// Optimized tensor-to-vector conversion
|
||||||
throw std::runtime_error("No models have been trained");
|
std::vector<int> result = platform::TensorUtils::to_vector<int>(predictions_tensor);
|
||||||
}
|
return result;
|
||||||
|
|
||||||
// x should be a 1D tensor with n features
|
|
||||||
if (x.size(0) != n) {
|
|
||||||
throw std::runtime_error("Input sample has wrong number of features. Expected " +
|
|
||||||
std::to_string(n) + " but got " + std::to_string(x.size(0)));
|
|
||||||
}
|
|
||||||
|
|
||||||
// Initialize class votes (same logic as predictSample)
|
|
||||||
std::vector<double> class_votes(n_classes, 0.0);
|
|
||||||
|
|
||||||
// Accumulate weighted votes from all estimators (SAMME voting)
|
|
||||||
double total_alpha = 0.0;
|
|
||||||
for (size_t i = 0; i < models.size(); i++) {
|
|
||||||
if (alphas[i] <= 0) continue; // Skip estimators with zero or negative weight
|
|
||||||
|
|
||||||
try {
|
|
||||||
// Get class prediction from this estimator (not probabilities!)
|
|
||||||
int predicted_class = static_cast<DecisionTree*>(models[i].get())->predictSample(x);
|
|
||||||
|
|
||||||
// Add weighted vote for this class (SAMME algorithm)
|
|
||||||
if (predicted_class >= 0 && predicted_class < n_classes) {
|
|
||||||
class_votes[predicted_class] += alphas[i];
|
|
||||||
total_alpha += alphas[i];
|
|
||||||
}
|
|
||||||
}
|
|
||||||
catch (const std::exception& e) {
|
|
||||||
std::cerr << "Error in estimator " << i << ": " << e.what() << std::endl;
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Convert votes to probabilities
|
|
||||||
torch::Tensor class_probs = torch::zeros({ n_classes }, torch::kFloat);
|
|
||||||
|
|
||||||
if (total_alpha > 0) {
|
|
||||||
// Normalize votes to get probabilities
|
|
||||||
for (int j = 0; j < n_classes; j++) {
|
|
||||||
class_probs[j] = static_cast<float>(class_votes[j] / total_alpha);
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
// If no valid estimators, return uniform distribution
|
|
||||||
class_probs.fill_(1.0f / n_classes);
|
|
||||||
}
|
|
||||||
|
|
||||||
// Ensure probabilities are valid (they should be already, but just in case)
|
|
||||||
class_probs = torch::clamp(class_probs, 0.0f, 1.0f);
|
|
||||||
|
|
||||||
// Verify they sum to 1 (they should, but normalize if needed due to floating point errors)
|
|
||||||
float sum_probs = torch::sum(class_probs).item<float>();
|
|
||||||
if (sum_probs > 1e-15f) {
|
|
||||||
class_probs = class_probs / sum_probs;
|
|
||||||
} else {
|
|
||||||
class_probs.fill_(1.0f / n_classes);
|
|
||||||
}
|
|
||||||
|
|
||||||
return class_probs;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
} // namespace bayesnet
|
} // namespace bayesnet
|
@@ -14,7 +14,7 @@
|
|||||||
namespace bayesnet {
|
namespace bayesnet {
|
||||||
class AdaBoost : public Ensemble {
|
class AdaBoost : public Ensemble {
|
||||||
public:
|
public:
|
||||||
explicit AdaBoost(int n_estimators = 50, int max_depth = 1);
|
explicit AdaBoost(int n_estimators = 100, int max_depth = 1);
|
||||||
virtual ~AdaBoost() = default;
|
virtual ~AdaBoost() = default;
|
||||||
|
|
||||||
// Override base class methods
|
// Override base class methods
|
||||||
@@ -38,7 +38,7 @@ namespace bayesnet {
|
|||||||
torch::Tensor predict(torch::Tensor& X) override;
|
torch::Tensor predict(torch::Tensor& X) override;
|
||||||
std::vector<int> predict(std::vector<std::vector<int>>& X) override;
|
std::vector<int> predict(std::vector<std::vector<int>>& X) override;
|
||||||
torch::Tensor predict_proba(torch::Tensor& X) override;
|
torch::Tensor predict_proba(torch::Tensor& X) override;
|
||||||
std::vector<std::vector<double>> predict_proba(std::vector<std::vector<int>>& X);
|
std::vector<std::vector<double>> predict_proba(std::vector<std::vector<int>>& X) override;
|
||||||
void setDebug(bool debug) { this->debug = debug; }
|
void setDebug(bool debug) { this->debug = debug; }
|
||||||
|
|
||||||
protected:
|
protected:
|
||||||
|
@@ -10,7 +10,7 @@
|
|||||||
#include <sstream>
|
#include <sstream>
|
||||||
#include <iomanip>
|
#include <iomanip>
|
||||||
#include <limits>
|
#include <limits>
|
||||||
#include "TensorUtils.hpp"
|
#include "common/TensorUtils.hpp"
|
||||||
|
|
||||||
namespace bayesnet {
|
namespace bayesnet {
|
||||||
|
|
||||||
|
@@ -40,7 +40,7 @@ namespace bayesnet {
|
|||||||
torch::Tensor predict(torch::Tensor& X) override;
|
torch::Tensor predict(torch::Tensor& X) override;
|
||||||
std::vector<int> predict(std::vector<std::vector<int>>& X) override;
|
std::vector<int> predict(std::vector<std::vector<int>>& X) override;
|
||||||
torch::Tensor predict_proba(torch::Tensor& X) override;
|
torch::Tensor predict_proba(torch::Tensor& X) override;
|
||||||
std::vector<std::vector<double>> predict_proba(std::vector<std::vector<int>>& X);
|
std::vector<std::vector<double>> predict_proba(std::vector<std::vector<int>>& X) override;
|
||||||
|
|
||||||
// Make predictions for a single sample
|
// Make predictions for a single sample
|
||||||
int predictSample(const torch::Tensor& x) const;
|
int predictSample(const torch::Tensor& x) const;
|
||||||
|
@@ -5,7 +5,7 @@
|
|||||||
// ***************************************************************
|
// ***************************************************************
|
||||||
|
|
||||||
#include "ExpClf.h"
|
#include "ExpClf.h"
|
||||||
#include "TensorUtils.hpp"
|
#include "common/TensorUtils.hpp"
|
||||||
|
|
||||||
namespace platform {
|
namespace platform {
|
||||||
ExpClf::ExpClf() : semaphore_{ CountingSemaphore::getInstance() }, Boost(false)
|
ExpClf::ExpClf() : semaphore_{ CountingSemaphore::getInstance() }, Boost(false)
|
||||||
|
@@ -5,7 +5,7 @@
|
|||||||
// ***************************************************************
|
// ***************************************************************
|
||||||
|
|
||||||
#include "ExpEnsemble.h"
|
#include "ExpEnsemble.h"
|
||||||
#include "TensorUtils.hpp"
|
#include "common/TensorUtils.hpp"
|
||||||
|
|
||||||
namespace platform {
|
namespace platform {
|
||||||
ExpEnsemble::ExpEnsemble() : semaphore_{ CountingSemaphore::getInstance() }, Boost(false)
|
ExpEnsemble::ExpEnsemble() : semaphore_{ CountingSemaphore::getInstance() }, Boost(false)
|
||||||
|
@@ -5,7 +5,7 @@
|
|||||||
// ***************************************************************
|
// ***************************************************************
|
||||||
|
|
||||||
#include "XA1DE.h"
|
#include "XA1DE.h"
|
||||||
#include "TensorUtils.hpp"
|
#include "common/TensorUtils.hpp"
|
||||||
|
|
||||||
namespace platform {
|
namespace platform {
|
||||||
void XA1DE::trainModel(const torch::Tensor& weights, const bayesnet::Smoothing_t smoothing)
|
void XA1DE::trainModel(const torch::Tensor& weights, const bayesnet::Smoothing_t smoothing)
|
||||||
|
@@ -10,7 +10,7 @@
|
|||||||
#include <tuple>
|
#include <tuple>
|
||||||
#include "XBAODE.h"
|
#include "XBAODE.h"
|
||||||
#include "XSpode.hpp"
|
#include "XSpode.hpp"
|
||||||
#include "TensorUtils.hpp"
|
#include "common/TensorUtils.hpp"
|
||||||
#include <loguru.hpp>
|
#include <loguru.hpp>
|
||||||
|
|
||||||
namespace platform {
|
namespace platform {
|
||||||
|
@@ -246,8 +246,8 @@ namespace platform {
|
|||||||
//
|
//
|
||||||
clf->fit(X_train, y_train, features, className, states, smooth_type);
|
clf->fit(X_train, y_train, features, className, states, smooth_type);
|
||||||
auto clf_notes = clf->getNotes();
|
auto clf_notes = clf->getNotes();
|
||||||
std::transform(clf_notes.begin(), clf_notes.end(), std::back_inserter(notes), [nfold](const std::string& note)
|
std::transform(clf_notes.begin(), clf_notes.end(), std::back_inserter(notes), [seed, nfold](const std::string& note)
|
||||||
{ return "Fold " + std::to_string(nfold) + ": " + note; });
|
{ return "Seed: " + std::to_string(seed) + " Fold: " + std::to_string(nfold) + ": " + note; });
|
||||||
nodes[item] = clf->getNumberOfNodes();
|
nodes[item] = clf->getNumberOfNodes();
|
||||||
edges[item] = clf->getNumberOfEdges();
|
edges[item] = clf->getNumberOfEdges();
|
||||||
num_states[item] = clf->getNumberOfStates();
|
num_states[item] = clf->getNumberOfStates();
|
||||||
|
@@ -3,6 +3,7 @@
|
|||||||
#include <numeric>
|
#include <numeric>
|
||||||
#include <utility>
|
#include <utility>
|
||||||
#include "RocAuc.h"
|
#include "RocAuc.h"
|
||||||
|
#include "common/TensorUtils.hpp" // tensorToVector
|
||||||
namespace platform {
|
namespace platform {
|
||||||
|
|
||||||
double RocAuc::compute(const torch::Tensor& y_proba, const torch::Tensor& labels)
|
double RocAuc::compute(const torch::Tensor& y_proba, const torch::Tensor& labels)
|
||||||
|
@@ -1,6 +1,6 @@
|
|||||||
#include <sstream>
|
#include <sstream>
|
||||||
#include "Scores.h"
|
#include "Scores.h"
|
||||||
#include "common/Utils.h" // tensorToVector
|
#include "common/TensorUtils.hpp" // tensorToVector
|
||||||
#include "common/Colors.h"
|
#include "common/Colors.h"
|
||||||
namespace platform {
|
namespace platform {
|
||||||
Scores::Scores(torch::Tensor& y_test, torch::Tensor& y_proba, int num_classes, std::vector<std::string> labels) : num_classes(num_classes), labels(labels), y_test(y_test), y_proba(y_proba)
|
Scores::Scores(torch::Tensor& y_test, torch::Tensor& y_proba, int num_classes, std::vector<std::string> labels) : num_classes(num_classes), labels(labels), y_test(y_test), y_proba(y_proba)
|
||||||
@@ -50,7 +50,7 @@ namespace platform {
|
|||||||
auto nClasses = num_classes;
|
auto nClasses = num_classes;
|
||||||
if (num_classes == 2)
|
if (num_classes == 2)
|
||||||
nClasses = 1;
|
nClasses = 1;
|
||||||
auto y_testv = tensorToVector<int>(y_test);
|
auto y_testv = TensorUtils::tensorToVector<int>(y_test);
|
||||||
std::vector<double> aucScores(nClasses, 0.0);
|
std::vector<double> aucScores(nClasses, 0.0);
|
||||||
std::vector<std::pair<double, int>> scoresAndLabels;
|
std::vector<std::pair<double, int>> scoresAndLabels;
|
||||||
for (size_t classIdx = 0; classIdx < nClasses; ++classIdx) {
|
for (size_t classIdx = 0; classIdx < nClasses; ++classIdx) {
|
||||||
|
@@ -54,10 +54,8 @@ namespace platform {
|
|||||||
}
|
}
|
||||||
void ExcelFile::setProperties(std::string title)
|
void ExcelFile::setProperties(std::string title)
|
||||||
{
|
{
|
||||||
char line[title.size() + 1];
|
|
||||||
strcpy(line, title.c_str());
|
|
||||||
lxw_doc_properties properties = {
|
lxw_doc_properties properties = {
|
||||||
.title = line,
|
.title = title.c_str(),
|
||||||
.subject = (char*)"Machine learning results",
|
.subject = (char*)"Machine learning results",
|
||||||
.author = (char*)"Ricardo Montañana Gómez",
|
.author = (char*)"Ricardo Montañana Gómez",
|
||||||
.manager = (char*)"Dr. J. A. Gámez, Dr. J. M. Puerta",
|
.manager = (char*)"Dr. J. A. Gámez, Dr. J. M. Puerta",
|
||||||
|
@@ -2,14 +2,7 @@ if(ENABLE_TESTING)
|
|||||||
set(TEST_PLATFORM "unit_tests_platform")
|
set(TEST_PLATFORM "unit_tests_platform")
|
||||||
include_directories(
|
include_directories(
|
||||||
${Platform_SOURCE_DIR}/src
|
${Platform_SOURCE_DIR}/src
|
||||||
${Platform_SOURCE_DIR}/lib/argparse/include
|
|
||||||
${Platform_SOURCE_DIR}/lib/mdlp/src
|
|
||||||
${Platform_SOURCE_DIR}/lib/Files
|
|
||||||
${Platform_SOURCE_DIR}/lib/json/include
|
|
||||||
${Platform_SOURCE_DIR}/lib/folding
|
|
||||||
${CMAKE_BINARY_DIR}/configured_files/include
|
${CMAKE_BINARY_DIR}/configured_files/include
|
||||||
${PyClassifiers_INCLUDE_DIRS}
|
|
||||||
${Bayesnet_INCLUDE_DIRS}
|
|
||||||
)
|
)
|
||||||
set(TEST_SOURCES_PLATFORM
|
set(TEST_SOURCES_PLATFORM
|
||||||
TestUtils.cpp TestPlatform.cpp TestResult.cpp TestScores.cpp TestDecisionTree.cpp TestAdaBoost.cpp
|
TestUtils.cpp TestPlatform.cpp TestResult.cpp TestScores.cpp TestDecisionTree.cpp TestAdaBoost.cpp
|
||||||
@@ -19,6 +12,7 @@ if(ENABLE_TESTING)
|
|||||||
${Platform_SOURCE_DIR}/src/experimental_clfs/AdaBoost.cpp
|
${Platform_SOURCE_DIR}/src/experimental_clfs/AdaBoost.cpp
|
||||||
)
|
)
|
||||||
add_executable(${TEST_PLATFORM} ${TEST_SOURCES_PLATFORM})
|
add_executable(${TEST_PLATFORM} ${TEST_SOURCES_PLATFORM})
|
||||||
target_link_libraries(${TEST_PLATFORM} PUBLIC "${TORCH_LIBRARIES}" fimdlp Catch2::Catch2WithMain bayesnet)
|
target_link_libraries(${TEST_PLATFORM} PUBLIC
|
||||||
|
torch::torch fimdlp:fimdlp Catch2::Catch2WithMain bayesnet::bayesnet pyclassifiers::pyclassifiers)
|
||||||
add_test(NAME ${TEST_PLATFORM} COMMAND ${TEST_PLATFORM})
|
add_test(NAME ${TEST_PLATFORM} COMMAND ${TEST_PLATFORM})
|
||||||
endif(ENABLE_TESTING)
|
endif(ENABLE_TESTING)
|
||||||
|
@@ -13,12 +13,13 @@
|
|||||||
#include <stdexcept>
|
#include <stdexcept>
|
||||||
#include "experimental_clfs/AdaBoost.h"
|
#include "experimental_clfs/AdaBoost.h"
|
||||||
#include "experimental_clfs/DecisionTree.h"
|
#include "experimental_clfs/DecisionTree.h"
|
||||||
#include "experimental_clfs/TensorUtils.hpp"
|
#include "common/TensorUtils.hpp"
|
||||||
#include "TestUtils.h"
|
#include "TestUtils.h"
|
||||||
|
|
||||||
using namespace bayesnet;
|
using namespace bayesnet;
|
||||||
using namespace Catch::Matchers;
|
using namespace Catch::Matchers;
|
||||||
|
|
||||||
|
static const bool DEBUG = false;
|
||||||
|
|
||||||
TEST_CASE("AdaBoost Construction", "[AdaBoost]")
|
TEST_CASE("AdaBoost Construction", "[AdaBoost]")
|
||||||
{
|
{
|
||||||
@@ -141,6 +142,7 @@ TEST_CASE("AdaBoost Basic Functionality", "[AdaBoost]")
|
|||||||
SECTION("Prediction with vector interface")
|
SECTION("Prediction with vector interface")
|
||||||
{
|
{
|
||||||
AdaBoost ada(10, 3);
|
AdaBoost ada(10, 3);
|
||||||
|
ada.setDebug(DEBUG); // Enable debug to investigate
|
||||||
ada.fit(X, y, features, className, states, Smoothing_t::NONE);
|
ada.fit(X, y, features, className, states, Smoothing_t::NONE);
|
||||||
|
|
||||||
auto predictions = ada.predict(X);
|
auto predictions = ada.predict(X);
|
||||||
@@ -159,6 +161,7 @@ TEST_CASE("AdaBoost Basic Functionality", "[AdaBoost]")
|
|||||||
SECTION("Probability predictions with vector interface")
|
SECTION("Probability predictions with vector interface")
|
||||||
{
|
{
|
||||||
AdaBoost ada(10, 3);
|
AdaBoost ada(10, 3);
|
||||||
|
ada.setDebug(DEBUG); // ENABLE DEBUG HERE TOO
|
||||||
ada.fit(X, y, features, className, states, Smoothing_t::NONE);
|
ada.fit(X, y, features, className, states, Smoothing_t::NONE);
|
||||||
|
|
||||||
auto proba = ada.predict_proba(X);
|
auto proba = ada.predict_proba(X);
|
||||||
@@ -183,8 +186,16 @@ TEST_CASE("AdaBoost Basic Functionality", "[AdaBoost]")
|
|||||||
correct++;
|
correct++;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
INFO("Probability test - Sample " << i << ": pred=" << pred << ", probs=[" << p[0] << "," << p[1] << "], expected_from_probs=" << predicted_class);
|
||||||
|
|
||||||
|
// Handle ties
|
||||||
|
if (std::abs(p[0] - p[1]) < 1e-10) {
|
||||||
|
INFO("Tie detected in probabilities");
|
||||||
|
// Either prediction is valid in case of tie
|
||||||
|
} else {
|
||||||
// Check that predict_proba matches the expected predict value
|
// Check that predict_proba matches the expected predict value
|
||||||
REQUIRE(pred == (p[0] > p[1] ? 0 : 1));
|
REQUIRE(pred == predicted_class);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
double accuracy = static_cast<double>(correct) / n_samples;
|
double accuracy = static_cast<double>(correct) / n_samples;
|
||||||
REQUIRE(accuracy > 0.99); // Should achieve good accuracy on this simple dataset
|
REQUIRE(accuracy > 0.99); // Should achieve good accuracy on this simple dataset
|
||||||
@@ -230,103 +241,50 @@ TEST_CASE("AdaBoost Tensor Interface", "[AdaBoost]")
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
TEST_CASE("AdaBoost on Iris Dataset", "[AdaBoost][iris]")
|
TEST_CASE("AdaBoost SAMME Algorithm Validation", "[AdaBoost]")
|
||||||
{
|
{
|
||||||
auto raw = RawDatasets("iris", true);
|
auto raw = RawDatasets("iris", true);
|
||||||
|
|
||||||
SECTION("Training with vector interface")
|
SECTION("Prediction consistency with probabilities")
|
||||||
{
|
{
|
||||||
AdaBoost ada(30, 3);
|
AdaBoost ada(15, 3);
|
||||||
|
ada.setDebug(DEBUG); // Enable debug for ALL instances
|
||||||
REQUIRE_NOTHROW(ada.fit(raw.Xv, raw.yv, raw.featuresv, raw.classNamev, raw.statesv, Smoothing_t::NONE));
|
|
||||||
|
|
||||||
auto predictions = ada.predict(raw.Xv);
|
|
||||||
REQUIRE(predictions.size() == raw.yv.size());
|
|
||||||
|
|
||||||
// Calculate accuracy
|
|
||||||
int correct = 0;
|
|
||||||
for (size_t i = 0; i < predictions.size(); i++) {
|
|
||||||
if (predictions[i] == raw.yv[i]) correct++;
|
|
||||||
}
|
|
||||||
double accuracy = static_cast<double>(correct) / raw.yv.size();
|
|
||||||
REQUIRE(accuracy > 0.85); // Should achieve good accuracy
|
|
||||||
|
|
||||||
// Test probability predictions
|
|
||||||
auto proba = ada.predict_proba(raw.Xv);
|
|
||||||
REQUIRE(proba.size() == raw.yv.size());
|
|
||||||
REQUIRE(proba[0].size() == 3); // Three classes
|
|
||||||
|
|
||||||
// Verify estimator weights and errors
|
|
||||||
auto weights = ada.getEstimatorWeights();
|
|
||||||
auto errors = ada.getTrainingErrors();
|
|
||||||
|
|
||||||
REQUIRE(weights.size() == errors.size());
|
|
||||||
REQUIRE(weights.size() > 0);
|
|
||||||
|
|
||||||
// All weights should be positive (for non-zero error estimators)
|
|
||||||
for (double w : weights) {
|
|
||||||
REQUIRE(w >= 0.0);
|
|
||||||
}
|
|
||||||
|
|
||||||
// All errors should be less than 0.5 (better than random)
|
|
||||||
for (double e : errors) {
|
|
||||||
REQUIRE(e < 0.5);
|
|
||||||
REQUIRE(e >= 0.0);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
SECTION("Different number of estimators")
|
|
||||||
{
|
|
||||||
std::vector<int> n_estimators = { 5, 15, 25 };
|
|
||||||
|
|
||||||
for (int n_est : n_estimators) {
|
|
||||||
AdaBoost ada(n_est, 2);
|
|
||||||
ada.fit(raw.dataset, raw.featurest, raw.classNamet, raw.statest, Smoothing_t::NONE);
|
ada.fit(raw.dataset, raw.featurest, raw.classNamet, raw.statest, Smoothing_t::NONE);
|
||||||
|
|
||||||
auto predictions = ada.predict(raw.Xt);
|
auto predictions = ada.predict(raw.Xt);
|
||||||
REQUIRE(predictions.size(0) == raw.yt.size(0));
|
auto probabilities = ada.predict_proba(raw.Xt);
|
||||||
|
|
||||||
// Check that we don't exceed the specified number of estimators
|
REQUIRE(predictions.size(0) == probabilities.size(0));
|
||||||
auto weights = ada.getEstimatorWeights();
|
REQUIRE(probabilities.size(1) == 3); // Three classes in Iris
|
||||||
REQUIRE(static_cast<int>(weights.size()) <= n_est);
|
|
||||||
|
// For each sample, predicted class should correspond to highest probability
|
||||||
|
for (int i = 0; i < predictions.size(0); i++) {
|
||||||
|
int predicted_class = predictions[i].item<int>();
|
||||||
|
auto probs = probabilities[i];
|
||||||
|
|
||||||
|
// Find class with highest probability
|
||||||
|
auto max_prob_idx = torch::argmax(probs).item<int>();
|
||||||
|
|
||||||
|
// Predicted class should match class with highest probability
|
||||||
|
REQUIRE(predicted_class == max_prob_idx);
|
||||||
|
|
||||||
|
// Probabilities should sum to 1
|
||||||
|
double sum_probs = torch::sum(probs).item<double>();
|
||||||
|
REQUIRE(sum_probs == Catch::Approx(1.0).epsilon(1e-6));
|
||||||
|
|
||||||
|
// All probabilities should be non-negative
|
||||||
|
for (int j = 0; j < 3; j++) {
|
||||||
|
REQUIRE(probs[j].item<double>() >= 0.0);
|
||||||
|
REQUIRE(probs[j].item<double>() <= 1.0);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
SECTION("Different base estimator depths")
|
SECTION("Weighted voting verification")
|
||||||
{
|
{
|
||||||
std::vector<int> depths = { 1, 2, 4 };
|
// Simple dataset where we can verify the weighted voting
|
||||||
|
|
||||||
for (int depth : depths) {
|
|
||||||
AdaBoost ada(15, depth);
|
|
||||||
ada.fit(raw.dataset, raw.featurest, raw.classNamet, raw.statest, Smoothing_t::NONE);
|
|
||||||
|
|
||||||
auto predictions = ada.predict(raw.Xt);
|
|
||||||
REQUIRE(predictions.size(0) == raw.yt.size(0));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
TEST_CASE("AdaBoost Edge Cases", "[AdaBoost]")
|
|
||||||
{
|
|
||||||
auto raw = RawDatasets("iris", true);
|
|
||||||
|
|
||||||
SECTION("Single estimator (depth 1 stump)")
|
|
||||||
{
|
|
||||||
AdaBoost ada(1, 1); // Single decision stump
|
|
||||||
ada.fit(raw.dataset, raw.featurest, raw.classNamet, raw.statest, Smoothing_t::NONE);
|
|
||||||
|
|
||||||
auto predictions = ada.predict(raw.Xt);
|
|
||||||
REQUIRE(predictions.size(0) == raw.yt.size(0));
|
|
||||||
|
|
||||||
auto weights = ada.getEstimatorWeights();
|
|
||||||
REQUIRE(weights.size() == 1);
|
|
||||||
}
|
|
||||||
|
|
||||||
SECTION("Perfect classifier scenario")
|
|
||||||
{
|
|
||||||
// Create a perfectly separable dataset
|
|
||||||
std::vector<std::vector<int>> X = { {0,0,1,1}, {0,1,0,1} };
|
std::vector<std::vector<int>> X = { {0,0,1,1}, {0,1,0,1} };
|
||||||
std::vector<int> y = { 0, 0, 1, 1 };
|
std::vector<int> y = { 0, 1, 1, 0 };
|
||||||
std::vector<std::string> features = { "f1", "f2" };
|
std::vector<std::string> features = { "f1", "f2" };
|
||||||
std::string className = "class";
|
std::string className = "class";
|
||||||
std::map<std::string, std::vector<int>> states;
|
std::map<std::string, std::vector<int>> states;
|
||||||
@@ -334,191 +292,61 @@ TEST_CASE("AdaBoost Edge Cases", "[AdaBoost]")
|
|||||||
states["f2"] = { 0, 1 };
|
states["f2"] = { 0, 1 };
|
||||||
states["class"] = { 0, 1 };
|
states["class"] = { 0, 1 };
|
||||||
|
|
||||||
AdaBoost ada(10, 3);
|
|
||||||
ada.fit(X, y, features, className, states, Smoothing_t::NONE);
|
|
||||||
|
|
||||||
auto predictions = ada.predict(X);
|
|
||||||
REQUIRE(predictions.size() == 4);
|
|
||||||
|
|
||||||
// Should achieve perfect accuracy
|
|
||||||
int correct = 0;
|
|
||||||
for (size_t i = 0; i < predictions.size(); i++) {
|
|
||||||
if (predictions[i] == y[i]) correct++;
|
|
||||||
}
|
|
||||||
REQUIRE(correct == 4);
|
|
||||||
|
|
||||||
// Should stop early due to perfect classification
|
|
||||||
auto errors = ada.getTrainingErrors();
|
|
||||||
if (errors.size() > 0) {
|
|
||||||
REQUIRE(errors.back() < 1e-10); // Very low error
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
SECTION("Small dataset")
|
|
||||||
{
|
|
||||||
// Very small dataset
|
|
||||||
std::vector<std::vector<int>> X = { {0,1}, {1,0} };
|
|
||||||
std::vector<int> y = { 0, 1 };
|
|
||||||
std::vector<std::string> features = { "f1", "f2" };
|
|
||||||
std::string className = "class";
|
|
||||||
std::map<std::string, std::vector<int>> states;
|
|
||||||
states["f1"] = { 0, 1 };
|
|
||||||
states["f2"] = { 0, 1 };
|
|
||||||
states["class"] = { 0, 1 };
|
|
||||||
|
|
||||||
AdaBoost ada(5, 1);
|
|
||||||
REQUIRE_NOTHROW(ada.fit(X, y, features, className, states, Smoothing_t::NONE));
|
|
||||||
|
|
||||||
auto predictions = ada.predict(X);
|
|
||||||
REQUIRE(predictions.size() == 2);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
TEST_CASE("AdaBoost Graph Visualization", "[AdaBoost]")
|
|
||||||
{
|
|
||||||
// Simple dataset for visualization
|
|
||||||
std::vector<std::vector<int>> X = { {0,0,1,1}, {0,1,0,1} };
|
|
||||||
std::vector<int> y = { 0, 1, 1, 0 }; // XOR pattern
|
|
||||||
std::vector<std::string> features = { "x1", "x2" };
|
|
||||||
std::string className = "xor";
|
|
||||||
std::map<std::string, std::vector<int>> states;
|
|
||||||
states["x1"] = { 0, 1 };
|
|
||||||
states["x2"] = { 0, 1 };
|
|
||||||
states["xor"] = { 0, 1 };
|
|
||||||
|
|
||||||
SECTION("Graph generation")
|
|
||||||
{
|
|
||||||
AdaBoost ada(5, 2);
|
AdaBoost ada(5, 2);
|
||||||
|
ada.setDebug(DEBUG); // Enable debug for detailed logging
|
||||||
ada.fit(X, y, features, className, states, Smoothing_t::NONE);
|
ada.fit(X, y, features, className, states, Smoothing_t::NONE);
|
||||||
|
|
||||||
auto graph_lines = ada.graph();
|
INFO("=== Final test verification ===");
|
||||||
|
auto predictions = ada.predict(X);
|
||||||
|
auto probabilities = ada.predict_proba(X);
|
||||||
|
auto alphas = ada.getEstimatorWeights();
|
||||||
|
|
||||||
REQUIRE(graph_lines.size() > 2);
|
INFO("Training info:");
|
||||||
REQUIRE(graph_lines.front() == "digraph AdaBoost {");
|
for (size_t i = 0; i < alphas.size(); i++) {
|
||||||
REQUIRE(graph_lines.back() == "}");
|
INFO(" Model " << i << ": alpha=" << alphas[i]);
|
||||||
|
|
||||||
// Should contain base estimator references
|
|
||||||
bool has_estimators = false;
|
|
||||||
for (const auto& line : graph_lines) {
|
|
||||||
if (line.find("Estimator") != std::string::npos) {
|
|
||||||
has_estimators = true;
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
REQUIRE(has_estimators);
|
|
||||||
|
|
||||||
// Should contain alpha values
|
|
||||||
bool has_alpha = false;
|
|
||||||
for (const auto& line : graph_lines) {
|
|
||||||
if (line.find("α") != std::string::npos || line.find("alpha") != std::string::npos) {
|
|
||||||
has_alpha = true;
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
REQUIRE(has_alpha);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
SECTION("Graph with title")
|
REQUIRE(predictions.size() == 4);
|
||||||
|
REQUIRE(probabilities.size() == 4);
|
||||||
|
REQUIRE(probabilities[0].size() == 2); // Two classes
|
||||||
|
REQUIRE(alphas.size() > 0);
|
||||||
|
|
||||||
|
// Verify that estimator weights are reasonable
|
||||||
|
for (double alpha : alphas) {
|
||||||
|
REQUIRE(alpha >= 0.0); // Alphas should be non-negative
|
||||||
|
}
|
||||||
|
|
||||||
|
// Verify prediction-probability consistency with detailed logging
|
||||||
|
for (size_t i = 0; i < predictions.size(); i++) {
|
||||||
|
int pred = predictions[i];
|
||||||
|
auto probs = probabilities[i];
|
||||||
|
|
||||||
|
INFO("Final check - Sample " << i << ": predicted=" << pred << ", probabilities=[" << probs[0] << "," << probs[1] << "]");
|
||||||
|
|
||||||
|
// Handle the case where probabilities are exactly equal (tie)
|
||||||
|
if (std::abs(probs[0] - probs[1]) < 1e-10) {
|
||||||
|
INFO("Tie detected in probabilities - either prediction is valid");
|
||||||
|
REQUIRE((pred == 0 || pred == 1));
|
||||||
|
} else {
|
||||||
|
// Normal case - prediction should match max probability
|
||||||
|
int expected_pred = (probs[0] > probs[1]) ? 0 : 1;
|
||||||
|
INFO("Expected prediction based on probs: " << expected_pred);
|
||||||
|
REQUIRE(pred == expected_pred);
|
||||||
|
}
|
||||||
|
|
||||||
|
REQUIRE(probs[0] + probs[1] == Catch::Approx(1.0).epsilon(1e-6));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
SECTION("Empty models edge case")
|
||||||
{
|
{
|
||||||
AdaBoost ada(3, 1);
|
AdaBoost ada(1, 1);
|
||||||
ada.fit(X, y, features, className, states, Smoothing_t::NONE);
|
ada.setDebug(DEBUG); // Enable debug for ALL instances
|
||||||
|
|
||||||
auto graph_lines = ada.graph("XOR AdaBoost");
|
// Try to predict before fitting
|
||||||
|
std::vector<std::vector<int>> X = { {0}, {1} };
|
||||||
bool has_title = false;
|
REQUIRE_THROWS_WITH(ada.predict(X), ContainsSubstring("not been fitted"));
|
||||||
for (const auto& line : graph_lines) {
|
REQUIRE_THROWS_WITH(ada.predict_proba(X), ContainsSubstring("not been fitted"));
|
||||||
if (line.find("label=\"XOR AdaBoost\"") != std::string::npos) {
|
|
||||||
has_title = true;
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
REQUIRE(has_title);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
TEST_CASE("AdaBoost with Weights", "[AdaBoost]")
|
|
||||||
{
|
|
||||||
auto raw = RawDatasets("iris", true);
|
|
||||||
|
|
||||||
SECTION("Uniform weights")
|
|
||||||
{
|
|
||||||
AdaBoost ada(20, 3);
|
|
||||||
ada.fit(raw.dataset, raw.featurest, raw.classNamet, raw.statest, raw.weights, Smoothing_t::NONE);
|
|
||||||
|
|
||||||
auto predictions = ada.predict(raw.Xt);
|
|
||||||
REQUIRE(predictions.size(0) == raw.yt.size(0));
|
|
||||||
|
|
||||||
auto weights = ada.getEstimatorWeights();
|
|
||||||
REQUIRE(weights.size() > 0);
|
|
||||||
}
|
|
||||||
|
|
||||||
SECTION("Non-uniform weights")
|
|
||||||
{
|
|
||||||
auto weights = torch::ones({ raw.nSamples });
|
|
||||||
weights.index({ torch::indexing::Slice(0, 50) }) *= 3.0; // Emphasize first class
|
|
||||||
weights = weights / weights.sum();
|
|
||||||
|
|
||||||
AdaBoost ada(15, 2);
|
|
||||||
ada.fit(raw.dataset, raw.featurest, raw.classNamet, raw.statest, weights, Smoothing_t::NONE);
|
|
||||||
|
|
||||||
auto predictions = ada.predict(raw.Xt);
|
|
||||||
REQUIRE(predictions.size(0) == raw.yt.size(0));
|
|
||||||
|
|
||||||
// Check that training completed successfully
|
|
||||||
auto estimator_weights = ada.getEstimatorWeights();
|
|
||||||
auto errors = ada.getTrainingErrors();
|
|
||||||
|
|
||||||
REQUIRE(estimator_weights.size() == errors.size());
|
|
||||||
REQUIRE(estimator_weights.size() > 0);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
TEST_CASE("AdaBoost Input Dimension Validation", "[AdaBoost]")
|
|
||||||
{
|
|
||||||
auto raw = RawDatasets("iris", true);
|
|
||||||
|
|
||||||
SECTION("Correct input dimensions")
|
|
||||||
{
|
|
||||||
AdaBoost ada(10, 2);
|
|
||||||
ada.fit(raw.dataset, raw.featurest, raw.classNamet, raw.statest, Smoothing_t::NONE);
|
|
||||||
|
|
||||||
// Test with correct tensor dimensions (features x samples)
|
|
||||||
REQUIRE_NOTHROW(ada.predict(raw.Xt));
|
|
||||||
REQUIRE_NOTHROW(ada.predict_proba(raw.Xt));
|
|
||||||
|
|
||||||
// Test with correct vector dimensions (features x samples)
|
|
||||||
REQUIRE_NOTHROW(ada.predict(raw.Xv));
|
|
||||||
REQUIRE_NOTHROW(ada.predict_proba(raw.Xv));
|
|
||||||
}
|
|
||||||
|
|
||||||
SECTION("Dimension consistency between interfaces")
|
|
||||||
{
|
|
||||||
AdaBoost ada(10, 2);
|
|
||||||
ada.fit(raw.dataset, raw.featurest, raw.classNamet, raw.statest, Smoothing_t::NONE);
|
|
||||||
|
|
||||||
// Get predictions from both interfaces
|
|
||||||
auto tensor_predictions = ada.predict(raw.Xt);
|
|
||||||
auto vector_predictions = ada.predict(raw.Xv);
|
|
||||||
|
|
||||||
// Should have same number of predictions
|
|
||||||
REQUIRE(tensor_predictions.size(0) == static_cast<int>(vector_predictions.size()));
|
|
||||||
|
|
||||||
// Test probability predictions
|
|
||||||
auto tensor_proba = ada.predict_proba(raw.Xt);
|
|
||||||
auto vector_proba = ada.predict_proba(raw.Xv);
|
|
||||||
|
|
||||||
REQUIRE(tensor_proba.size(0) == static_cast<int>(vector_proba.size()));
|
|
||||||
REQUIRE(tensor_proba.size(1) == static_cast<int>(vector_proba[0].size()));
|
|
||||||
|
|
||||||
// Verify predictions match between interfaces
|
|
||||||
for (int i = 0; i < tensor_predictions.size(0); i++) {
|
|
||||||
REQUIRE(tensor_predictions[i].item<int>() == vector_predictions[i]);
|
|
||||||
|
|
||||||
// Verify probabilities match between interfaces
|
|
||||||
for (int j = 0; j < tensor_proba.size(1); j++) {
|
|
||||||
REQUIRE(tensor_proba[i][j].item<double>() == Catch::Approx(vector_proba[i][j]).epsilon(1e-10));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -548,6 +376,7 @@ TEST_CASE("AdaBoost Debug - Simple Dataset Analysis", "[AdaBoost][debug]")
|
|||||||
SECTION("Debug training process")
|
SECTION("Debug training process")
|
||||||
{
|
{
|
||||||
AdaBoost ada(5, 3); // Few estimators for debugging
|
AdaBoost ada(5, 3); // Few estimators for debugging
|
||||||
|
ada.setDebug(DEBUG);
|
||||||
|
|
||||||
// This should work perfectly on this simple dataset
|
// This should work perfectly on this simple dataset
|
||||||
REQUIRE_NOTHROW(ada.fit(X, y, features, className, states, Smoothing_t::NONE));
|
REQUIRE_NOTHROW(ada.fit(X, y, features, className, states, Smoothing_t::NONE));
|
||||||
@@ -603,9 +432,16 @@ TEST_CASE("AdaBoost Debug - Simple Dataset Analysis", "[AdaBoost][debug]")
|
|||||||
|
|
||||||
// Predicted class should match highest probability
|
// Predicted class should match highest probability
|
||||||
int pred_class = predictions[i];
|
int pred_class = predictions[i];
|
||||||
|
|
||||||
|
// Handle ties
|
||||||
|
if (std::abs(p[0] - p[1]) < 1e-10) {
|
||||||
|
INFO("Tie detected - probabilities are equal");
|
||||||
|
REQUIRE((pred_class == 0 || pred_class == 1));
|
||||||
|
} else {
|
||||||
REQUIRE(pred_class == (p[0] > p[1] ? 0 : 1));
|
REQUIRE(pred_class == (p[0] > p[1] ? 0 : 1));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
}
|
||||||
|
|
||||||
SECTION("Compare with single DecisionTree")
|
SECTION("Compare with single DecisionTree")
|
||||||
{
|
{
|
||||||
@@ -621,6 +457,7 @@ TEST_CASE("AdaBoost Debug - Simple Dataset Analysis", "[AdaBoost][debug]")
|
|||||||
double tree_accuracy = static_cast<double>(tree_correct) / n_samples;
|
double tree_accuracy = static_cast<double>(tree_correct) / n_samples;
|
||||||
|
|
||||||
AdaBoost ada(5, 3);
|
AdaBoost ada(5, 3);
|
||||||
|
ada.setDebug(DEBUG);
|
||||||
ada.fit(X, y, features, className, states, Smoothing_t::NONE);
|
ada.fit(X, y, features, className, states, Smoothing_t::NONE);
|
||||||
auto ada_predictions = ada.predict(X);
|
auto ada_predictions = ada.predict(X);
|
||||||
|
|
||||||
@@ -639,95 +476,6 @@ TEST_CASE("AdaBoost Debug - Simple Dataset Analysis", "[AdaBoost][debug]")
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
TEST_CASE("AdaBoost SAMME Algorithm Validation", "[AdaBoost]")
|
|
||||||
{
|
|
||||||
auto raw = RawDatasets("iris", true);
|
|
||||||
|
|
||||||
SECTION("Prediction consistency with probabilities")
|
|
||||||
{
|
|
||||||
AdaBoost ada(15, 3);
|
|
||||||
ada.fit(raw.dataset, raw.featurest, raw.classNamet, raw.statest, Smoothing_t::NONE);
|
|
||||||
|
|
||||||
auto predictions = ada.predict(raw.Xt);
|
|
||||||
auto probabilities = ada.predict_proba(raw.Xt);
|
|
||||||
|
|
||||||
REQUIRE(predictions.size(0) == probabilities.size(0));
|
|
||||||
REQUIRE(probabilities.size(1) == 3); // Three classes in Iris
|
|
||||||
|
|
||||||
// For each sample, predicted class should correspond to highest probability
|
|
||||||
for (int i = 0; i < predictions.size(0); i++) {
|
|
||||||
int predicted_class = predictions[i].item<int>();
|
|
||||||
auto probs = probabilities[i];
|
|
||||||
|
|
||||||
// Find class with highest probability
|
|
||||||
auto max_prob_idx = torch::argmax(probs).item<int>();
|
|
||||||
|
|
||||||
// Predicted class should match class with highest probability
|
|
||||||
REQUIRE(predicted_class == max_prob_idx);
|
|
||||||
|
|
||||||
// Probabilities should sum to 1
|
|
||||||
double sum_probs = torch::sum(probs).item<double>();
|
|
||||||
REQUIRE(sum_probs == Catch::Approx(1.0).epsilon(1e-6));
|
|
||||||
|
|
||||||
// All probabilities should be non-negative
|
|
||||||
for (int j = 0; j < 3; j++) {
|
|
||||||
REQUIRE(probs[j].item<double>() >= 0.0);
|
|
||||||
REQUIRE(probs[j].item<double>() <= 1.0);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
SECTION("Weighted voting verification")
|
|
||||||
{
|
|
||||||
// Simple dataset where we can verify the weighted voting
|
|
||||||
std::vector<std::vector<int>> X = { {0,0,1,1}, {0,1,0,1} };
|
|
||||||
std::vector<int> y = { 0, 1, 1, 0 };
|
|
||||||
std::vector<std::string> features = { "f1", "f2" };
|
|
||||||
std::string className = "class";
|
|
||||||
std::map<std::string, std::vector<int>> states;
|
|
||||||
states["f1"] = { 0, 1 };
|
|
||||||
states["f2"] = { 0, 1 };
|
|
||||||
states["class"] = { 0, 1 };
|
|
||||||
|
|
||||||
AdaBoost ada(5, 2);
|
|
||||||
ada.fit(X, y, features, className, states, Smoothing_t::NONE);
|
|
||||||
|
|
||||||
auto predictions = ada.predict(X);
|
|
||||||
auto probabilities = ada.predict_proba(X);
|
|
||||||
auto alphas = ada.getEstimatorWeights();
|
|
||||||
|
|
||||||
REQUIRE(predictions.size() == 4);
|
|
||||||
REQUIRE(probabilities.size() == 4);
|
|
||||||
REQUIRE(probabilities[0].size() == 2); // Two classes
|
|
||||||
REQUIRE(alphas.size() > 0);
|
|
||||||
|
|
||||||
// Verify that estimator weights are reasonable
|
|
||||||
for (double alpha : alphas) {
|
|
||||||
REQUIRE(alpha >= 0.0); // Alphas should be non-negative
|
|
||||||
}
|
|
||||||
|
|
||||||
// Verify prediction-probability consistency
|
|
||||||
for (size_t i = 0; i < predictions.size(); i++) {
|
|
||||||
int pred = predictions[i];
|
|
||||||
auto probs = probabilities[i];
|
|
||||||
INFO("Sample " << i << ": predicted=" << pred
|
|
||||||
<< ", probabilities=[" << probs[0] << ", " << probs[1] << "]");
|
|
||||||
|
|
||||||
REQUIRE(pred == (probs[0] > probs[1] ? 0 : 1));
|
|
||||||
REQUIRE(probs[0] + probs[1] == Catch::Approx(1.0).epsilon(1e-6));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
SECTION("Empty models edge case")
|
|
||||||
{
|
|
||||||
AdaBoost ada(1, 1);
|
|
||||||
|
|
||||||
// Try to predict before fitting
|
|
||||||
std::vector<std::vector<int>> X = { {0}, {1} };
|
|
||||||
REQUIRE_THROWS_WITH(ada.predict(X), ContainsSubstring("not been fitted"));
|
|
||||||
REQUIRE_THROWS_WITH(ada.predict_proba(X), ContainsSubstring("not been fitted"));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
TEST_CASE("AdaBoost Predict-Proba Consistency Fix", "[AdaBoost][consistency]")
|
TEST_CASE("AdaBoost Predict-Proba Consistency Fix", "[AdaBoost][consistency]")
|
||||||
{
|
{
|
||||||
// Simple binary classification dataset
|
// Simple binary classification dataset
|
||||||
@@ -743,20 +491,31 @@ TEST_CASE("AdaBoost Predict-Proba Consistency Fix", "[AdaBoost][consistency]")
|
|||||||
SECTION("Binary classification consistency")
|
SECTION("Binary classification consistency")
|
||||||
{
|
{
|
||||||
AdaBoost ada(3, 2);
|
AdaBoost ada(3, 2);
|
||||||
ada.setDebug(true); // Enable debug output
|
ada.setDebug(DEBUG); // Enable debug output
|
||||||
ada.fit(X, y, features, className, states, Smoothing_t::NONE);
|
ada.fit(X, y, features, className, states, Smoothing_t::NONE);
|
||||||
|
|
||||||
|
INFO("=== Debugging predict vs predict_proba consistency ===");
|
||||||
|
|
||||||
|
// Get training info
|
||||||
|
auto alphas = ada.getEstimatorWeights();
|
||||||
|
auto errors = ada.getTrainingErrors();
|
||||||
|
|
||||||
|
INFO("Training completed:");
|
||||||
|
INFO(" Number of models: " << alphas.size());
|
||||||
|
for (size_t i = 0; i < alphas.size(); i++) {
|
||||||
|
INFO(" Model " << i << ": alpha=" << alphas[i] << ", error=" << errors[i]);
|
||||||
|
}
|
||||||
|
|
||||||
auto predictions = ada.predict(X);
|
auto predictions = ada.predict(X);
|
||||||
auto probabilities = ada.predict_proba(X);
|
auto probabilities = ada.predict_proba(X);
|
||||||
|
|
||||||
INFO("=== Debugging predict vs predict_proba consistency ===");
|
|
||||||
|
|
||||||
// Verify consistency for each sample
|
// Verify consistency for each sample
|
||||||
for (size_t i = 0; i < predictions.size(); i++) {
|
for (size_t i = 0; i < predictions.size(); i++) {
|
||||||
int predicted_class = predictions[i];
|
int predicted_class = predictions[i];
|
||||||
auto probs = probabilities[i];
|
auto probs = probabilities[i];
|
||||||
|
|
||||||
INFO("Sample " << i << ":");
|
INFO("Sample " << i << ":");
|
||||||
|
INFO(" Features: [" << X[0][i] << ", " << X[1][i] << "]");
|
||||||
INFO(" True class: " << y[i]);
|
INFO(" True class: " << y[i]);
|
||||||
INFO(" Predicted class: " << predicted_class);
|
INFO(" Predicted class: " << predicted_class);
|
||||||
INFO(" Probabilities: [" << probs[0] << ", " << probs[1] << "]");
|
INFO(" Probabilities: [" << probs[0] << ", " << probs[1] << "]");
|
||||||
@@ -765,7 +524,14 @@ TEST_CASE("AdaBoost Predict-Proba Consistency Fix", "[AdaBoost][consistency]")
|
|||||||
int max_prob_class = (probs[0] > probs[1]) ? 0 : 1;
|
int max_prob_class = (probs[0] > probs[1]) ? 0 : 1;
|
||||||
INFO(" Max prob class: " << max_prob_class);
|
INFO(" Max prob class: " << max_prob_class);
|
||||||
|
|
||||||
|
// Handle tie case (when probabilities are equal)
|
||||||
|
if (std::abs(probs[0] - probs[1]) < 1e-10) {
|
||||||
|
INFO(" Tie detected - probabilities are equal");
|
||||||
|
// In case of tie, either prediction is valid
|
||||||
|
REQUIRE((predicted_class == 0 || predicted_class == 1));
|
||||||
|
} else {
|
||||||
REQUIRE(predicted_class == max_prob_class);
|
REQUIRE(predicted_class == max_prob_class);
|
||||||
|
}
|
||||||
|
|
||||||
// Probabilities should sum to 1
|
// Probabilities should sum to 1
|
||||||
double sum_probs = probs[0] + probs[1];
|
double sum_probs = probs[0] + probs[1];
|
||||||
@@ -778,37 +544,4 @@ TEST_CASE("AdaBoost Predict-Proba Consistency Fix", "[AdaBoost][consistency]")
|
|||||||
REQUIRE(probs[1] <= 1.0);
|
REQUIRE(probs[1] <= 1.0);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
SECTION("Multi-class consistency")
|
|
||||||
{
|
|
||||||
auto raw = RawDatasets("iris", true);
|
|
||||||
|
|
||||||
AdaBoost ada(5, 2);
|
|
||||||
ada.fit(raw.dataset, raw.featurest, raw.classNamet, raw.statest, Smoothing_t::NONE);
|
|
||||||
|
|
||||||
auto predictions = ada.predict(raw.Xt);
|
|
||||||
auto probabilities = ada.predict_proba(raw.Xt);
|
|
||||||
|
|
||||||
// Check consistency for first 10 samples
|
|
||||||
for (int i = 0; i < std::min(static_cast<int64_t>(10), predictions.size(0)); i++) {
|
|
||||||
int predicted_class = predictions[i].item<int>();
|
|
||||||
auto probs = probabilities[i];
|
|
||||||
|
|
||||||
// Find class with maximum probability
|
|
||||||
auto max_prob_idx = torch::argmax(probs).item<int>();
|
|
||||||
|
|
||||||
INFO("Sample " << i << ":");
|
|
||||||
INFO(" Predicted class: " << predicted_class);
|
|
||||||
INFO(" Max prob class: " << max_prob_idx);
|
|
||||||
INFO(" Probabilities: [" << probs[0].item<float>() << ", "
|
|
||||||
<< probs[1].item<float>() << ", " << probs[2].item<float>() << "]");
|
|
||||||
|
|
||||||
// They must match
|
|
||||||
REQUIRE(predicted_class == max_prob_idx);
|
|
||||||
|
|
||||||
// Probabilities should sum to 1
|
|
||||||
double sum_probs = torch::sum(probs).item<double>();
|
|
||||||
REQUIRE(sum_probs == Catch::Approx(1.0).epsilon(1e-6));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
}
|
@@ -7,7 +7,7 @@
|
|||||||
#include <string>
|
#include <string>
|
||||||
#include "TestUtils.h"
|
#include "TestUtils.h"
|
||||||
#include "folding.hpp"
|
#include "folding.hpp"
|
||||||
#include <ArffFiles/ArffFiles.hpp>
|
#include <ArffFiles.hpp>
|
||||||
#include <bayesnet/classifiers/TAN.h>
|
#include <bayesnet/classifiers/TAN.h>
|
||||||
#include "config_platform.h"
|
#include "config_platform.h"
|
||||||
|
|
||||||
|
@@ -1,21 +0,0 @@
|
|||||||
{
|
|
||||||
"default-registry": {
|
|
||||||
"kind": "git",
|
|
||||||
"baseline": "760bfd0c8d7c89ec640aec4df89418b7c2745605",
|
|
||||||
"repository": "https://github.com/microsoft/vcpkg"
|
|
||||||
},
|
|
||||||
"registries": [
|
|
||||||
{
|
|
||||||
"kind": "git",
|
|
||||||
"repository": "https://github.com/rmontanana/vcpkg-stash",
|
|
||||||
"baseline": "1ea69243c0e8b0de77c9d1dd6e1d7593ae7f3627",
|
|
||||||
"packages": [
|
|
||||||
"arff-files",
|
|
||||||
"bayesnet",
|
|
||||||
"fimdlp",
|
|
||||||
"folding",
|
|
||||||
"libtorch-bin"
|
|
||||||
]
|
|
||||||
}
|
|
||||||
]
|
|
||||||
}
|
|
43
vcpkg.json
43
vcpkg.json
@@ -1,43 +0,0 @@
|
|||||||
{
|
|
||||||
"name": "platform",
|
|
||||||
"version-string": "1.1.0",
|
|
||||||
"dependencies": [
|
|
||||||
"arff-files",
|
|
||||||
"nlohmann-json",
|
|
||||||
"fimdlp",
|
|
||||||
"libtorch-bin",
|
|
||||||
"folding",
|
|
||||||
"catch2",
|
|
||||||
"argparse"
|
|
||||||
],
|
|
||||||
"overrides": [
|
|
||||||
{
|
|
||||||
"name": "arff-files",
|
|
||||||
"version": "1.1.0"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "fimdlp",
|
|
||||||
"version": "2.0.1"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "libtorch-bin",
|
|
||||||
"version": "2.7.0"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "folding",
|
|
||||||
"version": "1.1.1"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "argpase",
|
|
||||||
"version": "3.2"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "catch2",
|
|
||||||
"version": "3.8.1"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "nlohmann-json",
|
|
||||||
"version": "3.11.3"
|
|
||||||
}
|
|
||||||
]
|
|
||||||
}
|
|
Reference in New Issue
Block a user