Compare commits
17 Commits
4e18dc87be
...
main
Author | SHA1 | Date | |
---|---|---|---|
c3c580a611
|
|||
515455695b
|
|||
f68d216150
|
|||
b990684581
|
|||
5fd0ef692d
|
|||
dfcdadbf38
|
|||
613f4b6813
|
|||
dc324fe5f7
|
|||
9816896240
|
|||
a3f765ce3c
|
|||
3d814a79c6
|
|||
1ef7ca6180 | |||
9448a971e8
|
|||
24cef7496d
|
|||
a1a6d3d612
|
|||
dda9740e83
|
|||
41afa1b888
|
2
.gitignore
vendored
2
.gitignore
vendored
@@ -43,3 +43,5 @@ diagrams/html/**
|
||||
diagrams/latex/**
|
||||
.cache
|
||||
vcpkg_installed
|
||||
.claude/settings.local.json
|
||||
CMakeUserPresets.json
|
||||
|
93
CHANGELOG.md
Normal file
93
CHANGELOG.md
Normal file
@@ -0,0 +1,93 @@
|
||||
# Changelog
|
||||
|
||||
All notable changes to this project will be documented in this file.
|
||||
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Changed
|
||||
- **BREAKING**: Migrated dependency management from vcpkg to Conan
|
||||
- Updated build system to use Conan toolchain files instead of vcpkg
|
||||
- Updated `make init` command to use `conan install` instead of `vcpkg install`
|
||||
- Modified CMakeLists.txt to use Conan's find_package mechanism
|
||||
- Updated documentation in CLAUDE.md to reflect Conan usage
|
||||
|
||||
### Added
|
||||
- `conanfile.py` - Conan recipe for dependency management with all required dependencies
|
||||
- CMakeUserPresets.json (generated by Conan)
|
||||
- Support for Conan build profiles (Release/Debug)
|
||||
|
||||
### Removed
|
||||
- `vcpkg.json` - vcpkg manifest file
|
||||
- `vcpkg-configuration.json` - vcpkg registry configuration
|
||||
- vcpkg toolchain dependency in build system
|
||||
|
||||
### Notes
|
||||
- The migration maintains compatibility with existing make targets and workflow
|
||||
- All dependencies now managed through Conan package manager
|
||||
|
||||
## [1.1.0] - 2025-07-02
|
||||
|
||||
### Added
|
||||
- **AdaBoost Implementation**: Complete multi-class SAMME AdaBoost classifier with optimization
|
||||
- Optimized AdaBoostPredict with 100 estimators as default
|
||||
- Enhanced predictProbaSample functionality
|
||||
- Full predict_proba support for probabilistic predictions
|
||||
- **Decision Tree Classifier**: New base classifier implementation with comprehensive tests
|
||||
- **XA1DE Model Family**: Extended Averaged One-Dependence Estimators
|
||||
- XA1DE, XBAODE, XSPODE variants with threading support
|
||||
- Complete integration with memory optimization
|
||||
- Prior probability computation in prediction
|
||||
- **Wilcoxon Statistical Test**: Statistical significance testing for model comparison
|
||||
- **Folder Management**: Enhanced file organization with folder parameter support across tools
|
||||
- Added folder parameter to b_best, b_grid, b_main, and b_manage
|
||||
- **vcpkg Integration**: Package management system integration (now migrated to Conan)
|
||||
|
||||
### Enhanced
|
||||
- **Grid Search System**: Complete refactoring with MPI parallelization
|
||||
- Grid experiment functionality with conditional result saving
|
||||
- Fixed smoothing problems and dataset ordering
|
||||
- Enhanced reporting and summary generation
|
||||
- **Excel Reporting**: Advanced Excel export capabilities
|
||||
- ReportExcelCompared class for side-by-side result comparison
|
||||
- Enhanced formatting with colors and fixed headers
|
||||
- Automatic file opening after generation
|
||||
- **Results Management**: Comprehensive result handling and validation
|
||||
- JSON schema validation for result format integrity
|
||||
- Improved console reporting with classification reports
|
||||
- Pagination support for large result sets
|
||||
- **Statistical Analysis**: Enhanced statistical testing and reporting
|
||||
- AUC (Area Under Curve) computation and reporting
|
||||
- Confusion matrix generation and visualization
|
||||
- Classification reports with color coding
|
||||
|
||||
### Performance Improvements
|
||||
- Optimized AdaBoost training and prediction algorithms
|
||||
- Enhanced memory management in XA1DE implementations
|
||||
- Improved discretization algorithms with MDLP integration
|
||||
- Faster ROC-AUC computation for binary classification problems
|
||||
|
||||
### Developer Experience
|
||||
- **Testing Framework**: Comprehensive test suite with Catch2
|
||||
- **Build System**: Streamlined CMake configuration with dependency management
|
||||
- **Documentation**: Enhanced project documentation and build instructions
|
||||
- **Code Quality**: Refactored codebase with improved error handling and logging
|
||||
|
||||
### Bug Fixes
|
||||
- Fixed predict_proba implementations across multiple classifiers
|
||||
- Resolved grid search dataset ordering issues
|
||||
- Fixed Excel report formatting and column width problems
|
||||
- Corrected time output formatting in various tools
|
||||
- Fixed memory leaks and stability issues in model implementations
|
||||
|
||||
## [1.0.0] - 2024-01-09
|
||||
|
||||
### Initial Release
|
||||
- **Core Framework**: Machine learning experimentation platform for Bayesian Networks
|
||||
- **Basic Classifiers**: Initial set of Bayesian network classifiers
|
||||
- **Experiment Management**: Basic experiment orchestration and result storage
|
||||
- **Dataset Support**: ARFF file format support with discretization
|
||||
- **Build System**: CMake-based build system with external library integration
|
||||
- **Command Line Tools**: Initial versions of b_main, b_best, b_list utilities
|
139
CLAUDE.md
Normal file
139
CLAUDE.md
Normal file
@@ -0,0 +1,139 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## Project Overview
|
||||
|
||||
Platform is a C++ machine learning framework for running experiments with Bayesian Networks and other classifiers. It supports both research-focused experimental classifiers and production-ready models through a unified interface.
|
||||
|
||||
## Build System
|
||||
|
||||
The project uses CMake with Make as the primary build system:
|
||||
|
||||
- **Release build**: `make release` (creates `build_Release/` directory)
|
||||
- **Debug build**: `make debug` (creates `build_Debug/` directory with testing and coverage enabled)
|
||||
- **Install binaries**: `make install` (copies executables to `~/bin` by default)
|
||||
- **Clean project**: `make clean` (removes build directories)
|
||||
- **Initialize dependencies**: `make init` (runs conan install for both Release and Debug)
|
||||
|
||||
### Testing
|
||||
|
||||
- **Run tests**: `make test` (builds debug version and runs all tests)
|
||||
- **Coverage report**: `make coverage` (runs tests and generates coverage with gcovr)
|
||||
- **Single test with options**: `make test opt="-s"` (verbose) or `make test opt="-c='Test Name'"` (specific test)
|
||||
|
||||
### Build Targets
|
||||
|
||||
Main executables (built from `src/commands/`):
|
||||
- `b_main`: Main experiment runner
|
||||
- `b_grid`: Grid search over hyperparameters
|
||||
- `b_best`: Best results analysis and comparison
|
||||
- `b_list`: Dataset listing and properties
|
||||
- `b_manage`: Results management interface
|
||||
- `b_results`: Results processing
|
||||
|
||||
## Dependencies
|
||||
|
||||
The project uses Conan for package management with these key dependencies:
|
||||
- **libtorch**: PyTorch C++ backend for tensor operations
|
||||
- **nlohmann_json**: JSON processing
|
||||
- **catch2**: Unit testing framework
|
||||
- **cli11**: Command-line argument parsing (replacement for argparse)
|
||||
|
||||
Custom dependencies (not available in ConanCenter):
|
||||
- **fimdlp**: MDLP discretization library (needs manual integration)
|
||||
- **folding**: Cross-validation utilities (needs manual integration)
|
||||
- **arff-files**: ARFF dataset file handling (needs manual integration)
|
||||
|
||||
External dependencies (managed separately):
|
||||
- **BayesNet**: Core Bayesian network classifiers (from `../lib/`)
|
||||
- **PyClassifiers**: Python classifier wrappers (from `../lib/`)
|
||||
- **MPI**: Message Passing Interface for parallel processing
|
||||
- **Boost**: Python integration and utilities
|
||||
|
||||
**Note**: Some dependencies (fimdlp, folding, arff-files) are not available in ConanCenter and need to be:
|
||||
- Built as custom Conan packages, or
|
||||
- Integrated using CMake FetchContent, or
|
||||
- Built separately and found via find_package
|
||||
|
||||
## Architecture
|
||||
|
||||
### Core Components
|
||||
|
||||
**Experiment Framework** (`src/main/`):
|
||||
- `Experiment.cpp/h`: Main experiment orchestration
|
||||
- `Models.cpp/h`: Classifier factory and registration system
|
||||
- `Scores.cpp/h`: Performance metrics calculation
|
||||
- `HyperParameters.cpp/h`: Parameter management
|
||||
- `ArgumentsExperiment.cpp/h`: Command-line argument handling
|
||||
|
||||
**Data Handling** (`src/common/`):
|
||||
- `Dataset.cpp/h`: Individual dataset representation
|
||||
- `Datasets.cpp/h`: Dataset collection management
|
||||
- `Discretization.cpp/h`: Data discretization utilities
|
||||
|
||||
**Classifiers** (`src/experimental_clfs/`):
|
||||
- `AdaBoost.cpp/h`: Multi-class SAMME AdaBoost implementation
|
||||
- `DecisionTree.cpp/h`: Decision tree base classifier
|
||||
- `XA1DE.cpp/h`: Extended AODE variants
|
||||
- Experimental implementations of Bayesian network classifiers
|
||||
|
||||
**Grid Search** (`src/grid/`):
|
||||
- `GridSearch.cpp/h`: Hyperparameter optimization
|
||||
- `GridExperiment.cpp/h`: Grid search experiment management
|
||||
- Uses MPI for parallel hyperparameter evaluation
|
||||
|
||||
**Results & Reporting** (`src/results/`, `src/reports/`):
|
||||
- JSON-based result storage with schema validation
|
||||
- Excel export capabilities via libxlsxwriter
|
||||
- Console and paginated result display
|
||||
|
||||
### Model Registration System
|
||||
|
||||
The framework uses a factory pattern with automatic registration:
|
||||
- All classifiers inherit from `bayesnet::BaseClassifier`
|
||||
- Registration happens in `src/main/modelRegister.h`
|
||||
- Factory creates instances by string name via `Models::create()`
|
||||
|
||||
## Configuration
|
||||
|
||||
**Environment Configuration** (`.env` file):
|
||||
- `experiment`: Experiment name/type
|
||||
- `n_folds`: Cross-validation folds (default: 5)
|
||||
- `seeds`: Random seeds for reproducibility
|
||||
- `model`: Default classifier name
|
||||
- `score`: Primary evaluation metric
|
||||
- `platform`: System identifier for results
|
||||
|
||||
**Grid Search Configuration**:
|
||||
- `grid_<model_name>_input.json`: Hyperparameter search space
|
||||
- `grid_<model_name>_output.json`: Search results
|
||||
|
||||
## Data Format
|
||||
|
||||
**Dataset Requirements**:
|
||||
- ARFF format files in `datasets/` directory
|
||||
- `all.txt` file listing datasets: `<name>,<class_name>,<real_features>`
|
||||
- Supports both discrete and continuous features
|
||||
- Automatic discretization available via MDLP
|
||||
|
||||
**Experimental Data**:
|
||||
- Results stored in JSON format with versioned schemas
|
||||
- Test data in `tests/data/` for unit testing
|
||||
- Sample datasets: iris, diabetes, ecoli, glass, etc.
|
||||
|
||||
## Development Workflow
|
||||
|
||||
1. **Setup**: Run `make init` to install dependencies via Conan
|
||||
2. **Development**: Use `make debug` for development builds with testing
|
||||
3. **Testing**: Run `make test` after changes
|
||||
4. **Release**: Use `make release` for optimized builds
|
||||
5. **Experiments**: Use `.env` configuration and run `b_main` with appropriate flags
|
||||
|
||||
## Key Features
|
||||
|
||||
- **Multi-threaded**: Uses MPI for parallel grid search and experiments
|
||||
- **Cross-platform**: Supports Linux and macOS via vcpkg
|
||||
- **Extensible**: Easy classifier registration and integration
|
||||
- **Research-focused**: Designed for machine learning experimentation
|
||||
- **Visualization**: DOT graph generation for decision trees and networks
|
@@ -13,10 +13,10 @@ set(CMAKE_CXX_STANDARD 20)
|
||||
set(CMAKE_CXX_STANDARD_REQUIRED ON)
|
||||
set(CMAKE_CXX_EXTENSIONS OFF)
|
||||
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
|
||||
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
|
||||
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${TORCH_CXX_FLAGS}")
|
||||
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -pthread")
|
||||
set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -Ofast")
|
||||
set(CMAKE_CXX_FLAGS_DEBUG " ${CMAKE_CXX_FLAGS} -fprofile-arcs -ftest-coverage -O0 -g")
|
||||
set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -O3")
|
||||
|
||||
# Options
|
||||
# -------
|
||||
@@ -43,8 +43,6 @@ find_package(Boost 1.66.0 REQUIRED COMPONENTS python3 numpy3)
|
||||
|
||||
# # Python
|
||||
find_package(Python3 REQUIRED COMPONENTS Development)
|
||||
# # target_include_directories(MyTarget SYSTEM PRIVATE ${Python3_INCLUDE_DIRS})
|
||||
# message("Python_LIBRARIES=${Python_LIBRARIES}")
|
||||
|
||||
# # Boost Python
|
||||
# find_package(boost_python${Python3_VERSION_MAJOR}${Python3_VERSION_MINOR} CONFIG REQUIRED COMPONENTS python${Python3_VERSION_MAJOR}${Python3_VERSION_MINOR})
|
||||
@@ -62,35 +60,16 @@ endif()
|
||||
|
||||
# External libraries - dependencies of Platform
|
||||
# ---------------------------------------------
|
||||
|
||||
find_library(XLSXWRITER_LIB NAMES libxlsxwriter.dylib libxlsxwriter.so PATHS ${Platform_SOURCE_DIR}/lib/libxlsxwriter/lib)
|
||||
# find_path(XLSXWRITER_INCLUDE_DIR xlsxwriter.h)
|
||||
# find_library(XLSXWRITER_LIBRARY xlsxwriter)
|
||||
# message("XLSXWRITER_INCLUDE_DIR=${XLSXWRITER_INCLUDE_DIR}")
|
||||
# message("XLSXWRITER_LIBRARY=${XLSXWRITER_LIBRARY}")
|
||||
find_package(nlohmann_json CONFIG REQUIRED)
|
||||
find_package(argparse CONFIG REQUIRED)
|
||||
find_package(Torch CONFIG REQUIRED)
|
||||
find_package(arff-files CONFIG REQUIRED)
|
||||
find_package(fimdlp CONFIG REQUIRED)
|
||||
find_package(folding CONFIG REQUIRED)
|
||||
find_package(argparse CONFIG REQUIRED)
|
||||
find_package(nlohmann_json CONFIG REQUIRED)
|
||||
find_package(bayesnet CONFIG REQUIRED)
|
||||
find_package(pyclassifiers CONFIG REQUIRED)
|
||||
find_package(libxlsxwriter CONFIG REQUIRED)
|
||||
find_package(Boost REQUIRED COMPONENTS python)
|
||||
find_package(arff-files CONFIG REQUIRED)
|
||||
|
||||
# BayesNet
|
||||
find_library(bayesnet NAMES libbayesnet bayesnet libbayesnet.a PATHS ${Platform_SOURCE_DIR}/../lib/lib REQUIRED)
|
||||
find_path(Bayesnet_INCLUDE_DIRS REQUIRED NAMES bayesnet PATHS ${Platform_SOURCE_DIR}/../lib/include)
|
||||
add_library(bayesnet::bayesnet UNKNOWN IMPORTED)
|
||||
set_target_properties(bayesnet::bayesnet PROPERTIES
|
||||
IMPORTED_LOCATION ${bayesnet}
|
||||
INTERFACE_INCLUDE_DIRECTORIES ${Bayesnet_INCLUDE_DIRS})
|
||||
message(STATUS "BayesNet=${bayesnet}")
|
||||
message(STATUS "BayesNet_INCLUDE_DIRS=${Bayesnet_INCLUDE_DIRS}")
|
||||
|
||||
# PyClassifiers
|
||||
find_library(PyClassifiers NAMES libPyClassifiers PyClassifiers libPyClassifiers.a PATHS ${Platform_SOURCE_DIR}/../lib/lib REQUIRED)
|
||||
find_path(PyClassifiers_INCLUDE_DIRS REQUIRED NAMES pyclassifiers PATHS ${Platform_SOURCE_DIR}/../lib/include)
|
||||
message(STATUS "PyClassifiers=${PyClassifiers}")
|
||||
message(STATUS "PyClassifiers_INCLUDE_DIRS=${PyClassifiers_INCLUDE_DIRS}")
|
||||
|
||||
# Subdirectories
|
||||
# --------------
|
||||
@@ -105,14 +84,16 @@ file(GLOB Platform_SOURCES CONFIGURE_DEPENDS ${Platform_SOURCE_DIR}/src/*.cpp)
|
||||
# Testing
|
||||
# -------
|
||||
if (ENABLE_TESTING)
|
||||
enable_testing()
|
||||
MESSAGE("Testing enabled")
|
||||
set(CMAKE_CXX_FLAGS_DEBUG " ${CMAKE_CXX_FLAGS} -fprofile-arcs -ftest-coverage -O0 -g")
|
||||
enable_testing()
|
||||
find_package(Catch2 CONFIG REQUIRED)
|
||||
set(CODE_COVERAGE ON)
|
||||
include(CTest)
|
||||
add_subdirectory(tests)
|
||||
endif (ENABLE_TESTING)
|
||||
if (CODE_COVERAGE)
|
||||
include(CodeCoverage)
|
||||
MESSAGE("Code coverage enabled")
|
||||
include(CodeCoverage)
|
||||
SET(GCC_COVERAGE_LINK_FLAGS " ${GCC_COVERAGE_LINK_FLAGS} -lgcov --coverage")
|
||||
endif (CODE_COVERAGE)
|
||||
|
64
Makefile
64
Makefile
@@ -6,6 +6,13 @@ f_release = build_Release
|
||||
f_debug = build_Debug
|
||||
app_targets = b_best b_list b_main b_manage b_grid b_results
|
||||
test_targets = unit_tests_platform
|
||||
# Set the number of parallel jobs to the number of available processors minus 7
|
||||
CPUS := $(shell getconf _NPROCESSORS_ONLN 2>/dev/null \
|
||||
|| nproc --all 2>/dev/null \
|
||||
|| sysctl -n hw.ncpu)
|
||||
|
||||
# --- Your desired job count: CPUs – 7, but never less than 1 --------------
|
||||
JOBS := $(shell n=$(CPUS); [ $${n} -gt 7 ] && echo $$((n-7)) || echo 1)
|
||||
|
||||
define ClearTests
|
||||
@for t in $(test_targets); do \
|
||||
@@ -20,15 +27,36 @@ define ClearTests
|
||||
fi ;
|
||||
endef
|
||||
|
||||
define build_target
|
||||
@echo ">>> Building the project for $(1)..."
|
||||
@if [ -d $(2) ]; then rm -fr $(2); fi
|
||||
@conan install . --build=missing -of $(2) -s build_type=$(1)
|
||||
@cmake -S . -B $(2) -DCMAKE_TOOLCHAIN_FILE=$(2)/build/$(1)/generators/conan_toolchain.cmake -DCMAKE_BUILD_TYPE=$(1) -D$(3)
|
||||
@echo ">>> Will build using $(JOBS) parallel jobs"
|
||||
echo ">>> Done"
|
||||
endef
|
||||
|
||||
define compile_target
|
||||
@echo ">>> Compiling for $(1)..."
|
||||
if [ "$(3)" != "" ]; then \
|
||||
target="-t$(3)"; \
|
||||
else \
|
||||
target=""; \
|
||||
fi
|
||||
@cmake --build $(2) --config $(1) --parallel $(JOBS) $(target)
|
||||
@echo ">>> Done"
|
||||
endef
|
||||
|
||||
init: ## Initialize the project installing dependencies
|
||||
@echo ">>> Installing dependencies"
|
||||
@vcpkg install
|
||||
@echo ">>> Done";
|
||||
@echo ">>> Installing dependencies with Conan"
|
||||
@conan install . --output-folder=build --build=missing -s build_type=Release
|
||||
@conan install . --output-folder=build_debug --build=missing -s build_type=Debug
|
||||
@echo ">>> Done"
|
||||
|
||||
clean: ## Clean the project
|
||||
@echo ">>> Cleaning the project..."
|
||||
@if test -f CMakeCache.txt ; then echo "- Deleting CMakeCache.txt"; rm -f CMakeCache.txt; fi
|
||||
@for folder in $(f_release) $(f_debug) vpcpkg_installed install_test ; do \
|
||||
@for folder in $(f_release) $(f_debug) build build_debug install_test ; do \
|
||||
if test -d "$$folder" ; then \
|
||||
echo "- Deleting $$folder folder" ; \
|
||||
rm -rf "$$folder"; \
|
||||
@@ -45,11 +73,6 @@ setup: ## Install dependencies for tests and coverage
|
||||
pip install gcovr; \
|
||||
fi
|
||||
|
||||
dest ?= ${HOME}/bin
|
||||
main: ## Build only the b_main target
|
||||
@cmake --build $(f_release) -t b_main --parallel
|
||||
@cp $(f_release)/src/b_main $(dest)
|
||||
|
||||
dest ?= ${HOME}/bin
|
||||
install: ## Copy binary files to bin folder
|
||||
@echo "Destination folder: $(dest)"
|
||||
@@ -70,34 +93,27 @@ dependency: ## Create a dependency graph diagram of the project (build/dependenc
|
||||
cd $(f_debug) && cmake .. --graphviz=dependency.dot && dot -Tpng dependency.dot -o dependency.png
|
||||
|
||||
buildd: ## Build the debug targets
|
||||
@cmake --build $(f_debug) -t $(app_targets) PlatformSample --parallel
|
||||
@$(call compile_target,"Debug","$(f_debug)")
|
||||
|
||||
buildr: ## Build the release targets
|
||||
@cmake --build $(f_release) -t $(app_targets) --parallel
|
||||
@$(call compile_target,"Release","$(f_release)")
|
||||
|
||||
clang-uml: ## Create uml class and sequence diagrams
|
||||
clang-uml -p --add-compile-flag -I /usr/lib/gcc/x86_64-redhat-linux/8/include/
|
||||
|
||||
debug: ## Build a debug version of the project with BayesNet from vcpkg
|
||||
@echo ">>> Building Debug Platform...";
|
||||
@if [ -d ./$(f_debug) ]; then rm -rf ./$(f_debug); fi
|
||||
@mkdir $(f_debug);
|
||||
@cmake -S . -B $(f_debug) -D CMAKE_BUILD_TYPE=Debug -D ENABLE_TESTING=ON -D CODE_COVERAGE=ON -D CMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE=${VCPKG_ROOT}/scripts/buildsystems/vcpkg.cmake
|
||||
@echo ">>> Done";
|
||||
debug: ## Build a debug version of the project with Conan
|
||||
@$(call build_target,"Debug","$(f_debug)", "ENABLE_TESTING=ON")
|
||||
|
||||
release: ## Build a Release version of the project with Conan
|
||||
@$(call build_target,"Release","$(f_release)", "ENABLE_TESTING=OFF")
|
||||
|
||||
release: ## Build a Release version of the project with BayesNet from vcpkg
|
||||
@echo ">>> Building Release Platform...";
|
||||
@if [ -d ./$(f_release) ]; then rm -rf ./$(f_release); fi
|
||||
@mkdir $(f_release);
|
||||
@cmake -S . -B $(f_release) -D CMAKE_BUILD_TYPE=Release -D CMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE=${VCPKG_ROOT}/scripts/buildsystems/vcpkg.cmake
|
||||
@echo ">>> Done";
|
||||
|
||||
opt = ""
|
||||
test: ## Run tests (opt="-s") to verbose output the tests, (opt="-c='Test Maximum Spanning Tree'") to run only that section
|
||||
@echo ">>> Running Platform tests...";
|
||||
@$(MAKE) clean
|
||||
@$(MAKE) debug
|
||||
@cmake --build $(f_debug) -t $(test_targets) --parallel
|
||||
@$(call "Compile_target", "Debug", "$(f_debug)", $(test_targets))
|
||||
@for t in $(test_targets); do \
|
||||
if [ -f $(f_debug)/tests/$$t ]; then \
|
||||
cd $(f_debug)/tests ; \
|
||||
|
42
conanfile.py
Normal file
42
conanfile.py
Normal file
@@ -0,0 +1,42 @@
|
||||
from conan import ConanFile
|
||||
from conan.tools.cmake import CMakeToolchain, CMakeDeps, cmake_layout
|
||||
|
||||
|
||||
class PlatformConan(ConanFile):
|
||||
name = "platform"
|
||||
version = "1.1.0"
|
||||
|
||||
# Binary configuration
|
||||
settings = "os", "compiler", "build_type", "arch"
|
||||
|
||||
# Sources are located in the same place as this recipe, copy them to the recipe
|
||||
exports_sources = "CMakeLists.txt", "src/*", "tests/*", "config/*", "cmake/*"
|
||||
|
||||
def requirements(self):
|
||||
# Core dependencies from vcpkg.json
|
||||
self.requires("argparse/3.2")
|
||||
self.requires("libtorch/2.7.1")
|
||||
self.requires("nlohmann_json/3.11.3")
|
||||
self.requires("folding/1.1.2")
|
||||
self.requires("fimdlp/2.1.1")
|
||||
self.requires("arff-files/1.2.1")
|
||||
self.requires("bayesnet/1.2.1")
|
||||
self.requires("pyclassifiers/1.0.3")
|
||||
self.requires("libxlsxwriter/1.2.2")
|
||||
|
||||
def build_requirements(self):
|
||||
self.tool_requires("cmake/[>=3.30]")
|
||||
self.test_requires("catch2/3.8.1")
|
||||
|
||||
def layout(self):
|
||||
cmake_layout(self)
|
||||
|
||||
def generate(self):
|
||||
deps = CMakeDeps(self)
|
||||
deps.generate()
|
||||
tc = CMakeToolchain(self)
|
||||
tc.generate()
|
||||
|
||||
def configure(self):
|
||||
# C++20 requirement
|
||||
self.settings.compiler.cppstd = "20"
|
@@ -1,13 +1,8 @@
|
||||
include_directories(
|
||||
## Libs
|
||||
${Python3_INCLUDE_DIRS}
|
||||
${MPI_CXX_INCLUDE_DIRS}
|
||||
${TORCH_INCLUDE_DIRS}
|
||||
${CMAKE_BINARY_DIR}/configured_files/include
|
||||
${PyClassifiers_INCLUDE_DIRS}
|
||||
## Platform
|
||||
${Platform_SOURCE_DIR}/src
|
||||
${Platform_SOURCE_DIR}/results
|
||||
)
|
||||
|
||||
# b_best
|
||||
@@ -23,7 +18,7 @@ add_executable(
|
||||
experimental_clfs/DecisionTree.cpp
|
||||
experimental_clfs/AdaBoost.cpp
|
||||
)
|
||||
target_link_libraries(b_best Boost::boost "${PyClassifiers}" bayesnet::bayesnet fimdlp ${Python3_LIBRARIES} "${TORCH_LIBRARIES}" Boost::python Boost::numpy "${XLSXWRITER_LIB}")
|
||||
target_link_libraries(b_best Boost::boost pyclassifiers::pyclassifiers bayesnet::bayesnet argparse::argparse fimdlp::fimdlp ${Python3_LIBRARIES} torch::torch Boost::python Boost::numpy libxlsxwriter::libxlsxwriter)
|
||||
|
||||
# b_grid
|
||||
set(grid_sources GridSearch.cpp GridData.cpp GridExperiment.cpp GridBase.cpp )
|
||||
@@ -38,7 +33,7 @@ add_executable(b_grid commands/b_grid.cpp ${grid_sources}
|
||||
experimental_clfs/DecisionTree.cpp
|
||||
experimental_clfs/AdaBoost.cpp
|
||||
)
|
||||
target_link_libraries(b_grid ${MPI_CXX_LIBRARIES} "${PyClassifiers}" bayesnet::bayesnet fimdlp ${Python3_LIBRARIES} "${TORCH_LIBRARIES}" Boost::python Boost::numpy)
|
||||
target_link_libraries(b_grid ${MPI_CXX_LIBRARIES} pyclassifiers::pyclassifiers bayesnet::bayesnet argparse::argparse fimdlp::fimdlp ${Python3_LIBRARIES} torch::torch Boost::python Boost::numpy)
|
||||
|
||||
# b_list
|
||||
add_executable(b_list commands/b_list.cpp
|
||||
@@ -51,7 +46,7 @@ add_executable(b_list commands/b_list.cpp
|
||||
experimental_clfs/DecisionTree.cpp
|
||||
experimental_clfs/AdaBoost.cpp
|
||||
)
|
||||
target_link_libraries(b_list "${PyClassifiers}" bayesnet::bayesnet fimdlp ${Python3_LIBRARIES} "${TORCH_LIBRARIES}" Boost::python Boost::numpy "${XLSXWRITER_LIB}")
|
||||
target_link_libraries(b_list pyclassifiers::pyclassifiers bayesnet::bayesnet argparse::argparse fimdlp::fimdlp ${Python3_LIBRARIES} torch::torch Boost::python Boost::numpy libxlsxwriter::libxlsxwriter)
|
||||
|
||||
# b_main
|
||||
set(main_sources Experiment.cpp Models.cpp HyperParameters.cpp Scores.cpp ArgumentsExperiment.cpp)
|
||||
@@ -66,7 +61,7 @@ add_executable(b_main commands/b_main.cpp ${main_sources}
|
||||
experimental_clfs/DecisionTree.cpp
|
||||
experimental_clfs/AdaBoost.cpp
|
||||
)
|
||||
target_link_libraries(b_main PRIVATE nlohmann_json::nlohmann_json "${PyClassifiers}" bayesnet::bayesnet fimdlp ${Python3_LIBRARIES} "${TORCH_LIBRARIES}" Boost::python Boost::numpy)
|
||||
target_link_libraries(b_main PRIVATE nlohmann_json::nlohmann_json pyclassifiers::pyclassifiers bayesnet::bayesnet argparse::argparse fimdlp::fimdlp ${Python3_LIBRARIES} torch::torch Boost::python Boost::numpy)
|
||||
|
||||
# b_manage
|
||||
set(manage_sources ManageScreen.cpp OptionsMenu.cpp ResultsManager.cpp)
|
||||
@@ -78,7 +73,8 @@ add_executable(
|
||||
results/Result.cpp results/ResultsDataset.cpp results/ResultsDatasetConsole.cpp
|
||||
main/Scores.cpp
|
||||
)
|
||||
target_link_libraries(b_manage "${TORCH_LIBRARIES}" "${XLSXWRITER_LIB}" fimdlp bayesnet::bayesnet)
|
||||
target_link_libraries(b_manage torch::torch libxlsxwriter::libxlsxwriter fimdlp::fimdlp bayesnet::bayesnet argparse::argparse)
|
||||
|
||||
# b_results
|
||||
add_executable(b_results commands/b_results.cpp)
|
||||
target_link_libraries(b_results torch::torch libxlsxwriter::libxlsxwriter fimdlp::fimdlp bayesnet::bayesnet argparse::argparse)
|
||||
|
@@ -2,8 +2,8 @@
|
||||
#include <filesystem>
|
||||
#include <fstream>
|
||||
#include <vector>
|
||||
#include <argparse/argparse.hpp>
|
||||
#include <nlohmann/json.hpp>
|
||||
#include "nlohmann/json.hpp"
|
||||
#include "argparse/argparse.hpp"
|
||||
#include "common/Paths.h"
|
||||
#include "results/JsonValidator.h"
|
||||
#include "results/SchemaV1_0.h"
|
||||
|
@@ -1,4 +1,4 @@
|
||||
#include <ArffFiles/ArffFiles.hpp>
|
||||
#include <ArffFiles.hpp>
|
||||
#include <fstream>
|
||||
#include "Dataset.h"
|
||||
namespace platform {
|
||||
|
@@ -5,6 +5,15 @@
|
||||
namespace platform {
|
||||
class TensorUtils {
|
||||
public:
|
||||
template <typename T>
|
||||
static std::vector<T> tensorToVector(const torch::Tensor& tensor)
|
||||
{
|
||||
torch::Tensor contig_tensor = tensor.contiguous();
|
||||
auto num_elements = contig_tensor.numel();
|
||||
const T* tensor_data = contig_tensor.data_ptr<T>();
|
||||
std::vector<T> result(tensor_data, tensor_data + num_elements);
|
||||
return result;
|
||||
}
|
||||
static std::vector<std::vector<int>> to_matrix(const torch::Tensor& X)
|
||||
{
|
||||
// Ensure tensor is contiguous in memory
|
||||
@@ -53,7 +62,7 @@ namespace platform {
|
||||
torch::Tensor tensor = torch::empty({ static_cast<long>(rows), static_cast<long>(cols) }, torch::kInt64);
|
||||
for (size_t i = 0; i < rows; ++i) {
|
||||
for (size_t j = 0; j < cols; ++j) {
|
||||
tensor.index_put_({ static_cast<long>(i), static_cast<long>(j) }, data[i][j]);
|
||||
tensor.index_put_({static_cast<int64_t>(i), static_cast<int64_t>(j)}, torch::scalar_tensor(data[i][j]));
|
||||
}
|
||||
}
|
||||
return tensor;
|
@@ -6,17 +6,15 @@
|
||||
#include <string>
|
||||
#include <vector>
|
||||
#include <algorithm>
|
||||
#include <torch/torch.h>
|
||||
#include <cstdlib>
|
||||
#include <cmath>
|
||||
#include <ctime>
|
||||
#include <iomanip>
|
||||
#include <string.h>
|
||||
|
||||
extern char** environ;
|
||||
|
||||
namespace platform {
|
||||
template <typename T>
|
||||
std::vector<T> tensorToVector(const torch::Tensor& tensor)
|
||||
{
|
||||
torch::Tensor contig_tensor = tensor.contiguous();
|
||||
auto num_elements = contig_tensor.numel();
|
||||
const T* tensor_data = contig_tensor.data_ptr<T>();
|
||||
std::vector<T> result(tensor_data, tensor_data + num_elements);
|
||||
return result;
|
||||
}
|
||||
static std::string trim(const std::string& str)
|
||||
{
|
||||
std::string result = str;
|
||||
|
@@ -11,7 +11,15 @@
|
||||
#include <numeric>
|
||||
#include <sstream>
|
||||
#include <iomanip>
|
||||
#include "TensorUtils.hpp"
|
||||
#include "common/TensorUtils.hpp"
|
||||
|
||||
// Conditional debug macro for performance-critical sections
|
||||
#define DEBUG_LOG(condition, ...) \
|
||||
do { \
|
||||
if (__builtin_expect((condition), 0)) { \
|
||||
std::cout << __VA_ARGS__ << std::endl; \
|
||||
} \
|
||||
} while(0)
|
||||
|
||||
namespace bayesnet {
|
||||
|
||||
@@ -21,6 +29,8 @@ namespace bayesnet {
|
||||
validHyperparameters = { "n_estimators", "base_max_depth" };
|
||||
}
|
||||
|
||||
// Versión optimizada de buildModel - Reemplazar en AdaBoost.cpp:
|
||||
|
||||
void AdaBoost::buildModel(const torch::Tensor& weights)
|
||||
{
|
||||
// Initialize variables
|
||||
@@ -38,20 +48,23 @@ namespace bayesnet {
|
||||
|
||||
// If initial weights are provided, incorporate them
|
||||
if (weights.defined() && weights.numel() > 0) {
|
||||
sample_weights *= weights;
|
||||
if (weights.size(0) != n_samples) {
|
||||
throw std::runtime_error("weights must have the same length as number of samples");
|
||||
}
|
||||
sample_weights = weights.clone();
|
||||
normalizeWeights();
|
||||
}
|
||||
|
||||
// Debug information
|
||||
if (debug) {
|
||||
std::cout << "Starting AdaBoost training with " << n_estimators << " estimators" << std::endl;
|
||||
std::cout << "Number of classes: " << n_classes << std::endl;
|
||||
std::cout << "Number of features: " << n << std::endl;
|
||||
std::cout << "Number of samples: " << n_samples << std::endl;
|
||||
}
|
||||
// Conditional debug information (only when debug is enabled)
|
||||
DEBUG_LOG(debug, "Starting AdaBoost training with " << n_estimators << " estimators\n"
|
||||
<< "Number of classes: " << n_classes << "\n"
|
||||
<< "Number of features: " << n << "\n"
|
||||
<< "Number of samples: " << n_samples);
|
||||
|
||||
// Main AdaBoost training loop (SAMME algorithm)
|
||||
// (Stagewise Additive Modeling using a Multi - class Exponential loss)
|
||||
// Pre-compute random guess error threshold
|
||||
const double random_guess_error = 1.0 - (1.0 / static_cast<double>(n_classes));
|
||||
|
||||
// Main AdaBoost training loop (SAMME algorithm)
|
||||
for (int iter = 0; iter < n_estimators; ++iter) {
|
||||
// Train base estimator with current sample weights
|
||||
auto estimator = trainBaseEstimator(sample_weights);
|
||||
@@ -60,12 +73,9 @@ namespace bayesnet {
|
||||
double weighted_error = calculateWeightedError(estimator.get(), sample_weights);
|
||||
training_errors.push_back(weighted_error);
|
||||
|
||||
// Check if error is too high (worse than random guessing)
|
||||
double random_guess_error = 1.0 - (1.0 / n_classes);
|
||||
|
||||
// According to SAMME, we need error < random_guess_error
|
||||
if (weighted_error >= random_guess_error) {
|
||||
if (debug) std::cout << " Error >= random guess (" << random_guess_error << "), stopping" << std::endl;
|
||||
DEBUG_LOG(debug, "Error >= random guess (" << random_guess_error << "), stopping");
|
||||
// If only one estimator and it's worse than random, keep it with zero weight
|
||||
if (models.empty()) {
|
||||
models.push_back(std::move(estimator));
|
||||
@@ -76,7 +86,7 @@ namespace bayesnet {
|
||||
|
||||
// Check for perfect classification BEFORE calculating alpha
|
||||
if (weighted_error <= 1e-10) {
|
||||
if (debug) std::cout << " Perfect classification achieved (error=" << weighted_error << ")" << std::endl;
|
||||
DEBUG_LOG(debug, "Perfect classification achieved (error=" << weighted_error << ")");
|
||||
|
||||
// For perfect classification, use a large but finite alpha
|
||||
double alpha = 10.0 + std::log(static_cast<double>(n_classes - 1));
|
||||
@@ -85,12 +95,10 @@ namespace bayesnet {
|
||||
models.push_back(std::move(estimator));
|
||||
alphas.push_back(alpha);
|
||||
|
||||
if (debug) {
|
||||
std::cout << "Iteration " << iter << ":" << std::endl;
|
||||
std::cout << " Weighted error: " << weighted_error << std::endl;
|
||||
std::cout << " Alpha (finite): " << alpha << std::endl;
|
||||
std::cout << " Random guess error: " << random_guess_error << std::endl;
|
||||
}
|
||||
DEBUG_LOG(debug, "Iteration " << iter << ":\n"
|
||||
<< " Weighted error: " << weighted_error << "\n"
|
||||
<< " Alpha (finite): " << alpha << "\n"
|
||||
<< " Random guess error: " << random_guess_error);
|
||||
|
||||
break; // Stop training as we have a perfect classifier
|
||||
}
|
||||
@@ -115,18 +123,15 @@ namespace bayesnet {
|
||||
normalizeWeights();
|
||||
}
|
||||
|
||||
if (debug) {
|
||||
std::cout << "Iteration " << iter << ":" << std::endl;
|
||||
std::cout << " Weighted error: " << weighted_error << std::endl;
|
||||
std::cout << " Alpha: " << alpha << std::endl;
|
||||
std::cout << " Random guess error: " << random_guess_error << std::endl;
|
||||
std::cout << " Random guess error: " << random_guess_error << std::endl;
|
||||
}
|
||||
DEBUG_LOG(debug, "Iteration " << iter << ":\n"
|
||||
<< " Weighted error: " << weighted_error << "\n"
|
||||
<< " Alpha: " << alpha << "\n"
|
||||
<< " Random guess error: " << random_guess_error);
|
||||
}
|
||||
|
||||
// Set the number of models actually trained
|
||||
n_models = models.size();
|
||||
if (debug) std::cout << "AdaBoost training completed with " << n_models << " models" << std::endl;
|
||||
DEBUG_LOG(debug, "AdaBoost training completed with " << n_models << " models");
|
||||
}
|
||||
|
||||
void AdaBoost::trainModel(const torch::Tensor& weights, const Smoothing_t smoothing)
|
||||
@@ -152,44 +157,60 @@ namespace bayesnet {
|
||||
|
||||
double AdaBoost::calculateWeightedError(Classifier* estimator, const torch::Tensor& weights)
|
||||
{
|
||||
// Get features and labels from dataset
|
||||
// Get features and labels from dataset (avoid repeated indexing)
|
||||
auto X = dataset.index({ torch::indexing::Slice(0, dataset.size(0) - 1), torch::indexing::Slice() });
|
||||
auto y_true = dataset.index({ -1, torch::indexing::Slice() });
|
||||
|
||||
// Get predictions from the estimator
|
||||
auto y_pred = estimator->predict(X);
|
||||
|
||||
// Calculate weighted error
|
||||
auto incorrect = (y_pred != y_true).to(torch::kFloat);
|
||||
// Vectorized error calculation using PyTorch operations
|
||||
auto incorrect = (y_pred != y_true).to(torch::kDouble);
|
||||
|
||||
// Ensure weights are normalized
|
||||
auto normalized_weights = weights / weights.sum();
|
||||
// Direct dot product for weighted error (more efficient than sum)
|
||||
double weighted_error = torch::dot(incorrect, weights).item<double>();
|
||||
|
||||
// Calculate weighted error
|
||||
double weighted_error = torch::sum(incorrect * normalized_weights).item<double>();
|
||||
|
||||
return weighted_error;
|
||||
// Clamp to valid range in one operation
|
||||
return std::clamp(weighted_error, 1e-15, 1.0 - 1e-15);
|
||||
}
|
||||
|
||||
void AdaBoost::updateSampleWeights(Classifier* estimator, double alpha)
|
||||
{
|
||||
// Get predictions from the estimator
|
||||
// Get predictions from the estimator (reuse from calculateWeightedError if possible)
|
||||
auto X = dataset.index({ torch::indexing::Slice(0, dataset.size(0) - 1), torch::indexing::Slice() });
|
||||
auto y_true = dataset.index({ -1, torch::indexing::Slice() });
|
||||
auto y_pred = estimator->predict(X);
|
||||
|
||||
// Update weights according to SAMME algorithm
|
||||
// w_i = w_i * exp(alpha * I(y_i != y_pred_i))
|
||||
auto incorrect = (y_pred != y_true).to(torch::kFloat);
|
||||
// Vectorized weight update using PyTorch operations
|
||||
auto incorrect = (y_pred != y_true).to(torch::kDouble);
|
||||
|
||||
// Single vectorized operation instead of element-wise multiplication
|
||||
sample_weights *= torch::exp(alpha * incorrect);
|
||||
|
||||
// Vectorized clamping for numerical stability
|
||||
sample_weights = torch::clamp(sample_weights, 1e-15, 1e15);
|
||||
}
|
||||
|
||||
void AdaBoost::normalizeWeights()
|
||||
{
|
||||
// Normalize weights to sum to 1
|
||||
// Single-pass normalization using PyTorch operations
|
||||
double sum_weights = torch::sum(sample_weights).item<double>();
|
||||
if (sum_weights > 0) {
|
||||
|
||||
if (__builtin_expect(sum_weights <= 0, 0)) {
|
||||
// Reset to uniform if all weights are zero/negative (rare case)
|
||||
sample_weights = torch::ones_like(sample_weights) / sample_weights.size(0);
|
||||
} else {
|
||||
// Vectorized normalization
|
||||
sample_weights /= sum_weights;
|
||||
|
||||
// Vectorized minimum weight enforcement
|
||||
sample_weights = torch::clamp_min(sample_weights, 1e-15);
|
||||
|
||||
// Renormalize after clamping (if any weights were clamped)
|
||||
double new_sum = torch::sum(sample_weights).item<double>();
|
||||
if (new_sum != 1.0) {
|
||||
sample_weights /= new_sum;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -273,194 +294,199 @@ namespace bayesnet {
|
||||
Ensemble::setHyperparameters(hyperparameters);
|
||||
}
|
||||
|
||||
torch::Tensor AdaBoost::predict(torch::Tensor& X)
|
||||
int AdaBoost::predictSample(const torch::Tensor& x) const
|
||||
{
|
||||
if (!fitted) {
|
||||
// Early validation (keep essential checks only)
|
||||
if (!fitted || models.empty()) {
|
||||
throw std::runtime_error(CLASSIFIER_NOT_FITTED);
|
||||
}
|
||||
|
||||
if (models.empty()) {
|
||||
throw std::runtime_error("No models have been trained");
|
||||
// Pre-allocate and reuse memory
|
||||
static thread_local std::vector<double> class_votes_cache;
|
||||
if (class_votes_cache.size() != static_cast<size_t>(n_classes)) {
|
||||
class_votes_cache.resize(n_classes);
|
||||
}
|
||||
std::fill(class_votes_cache.begin(), class_votes_cache.end(), 0.0);
|
||||
|
||||
// Optimized voting loop - avoid exception handling in hot path
|
||||
for (size_t i = 0; i < models.size(); ++i) {
|
||||
double alpha = alphas[i];
|
||||
if (alpha <= 0 || !std::isfinite(alpha)) continue;
|
||||
|
||||
// Direct cast and call - avoid virtual dispatch overhead
|
||||
int predicted_class = static_cast<DecisionTree*>(models[i].get())->predictSample(x);
|
||||
|
||||
// Bounds check with branch prediction hint
|
||||
if (__builtin_expect(predicted_class >= 0 && predicted_class < n_classes, 1)) {
|
||||
class_votes_cache[predicted_class] += alpha;
|
||||
}
|
||||
}
|
||||
|
||||
// X should be (n_features, n_samples)
|
||||
if (X.size(0) != n) {
|
||||
throw std::runtime_error("Input has wrong number of features. Expected " +
|
||||
std::to_string(n) + " but got " + std::to_string(X.size(0)));
|
||||
// Fast argmax using iterators
|
||||
return std::distance(class_votes_cache.begin(),
|
||||
std::max_element(class_votes_cache.begin(), class_votes_cache.end()));
|
||||
}
|
||||
|
||||
torch::Tensor AdaBoost::predictProbaSample(const torch::Tensor& x) const
|
||||
{
|
||||
// Early validation
|
||||
if (!fitted || models.empty()) {
|
||||
throw std::runtime_error(CLASSIFIER_NOT_FITTED);
|
||||
}
|
||||
|
||||
int n_samples = X.size(1);
|
||||
torch::Tensor predictions = torch::zeros({ n_samples }, torch::kInt32);
|
||||
// Use stack allocation for small arrays (typical case: n_classes <= 32)
|
||||
constexpr int STACK_THRESHOLD = 32;
|
||||
double stack_votes[STACK_THRESHOLD];
|
||||
std::vector<double> heap_votes;
|
||||
double* class_votes;
|
||||
|
||||
for (int i = 0; i < n_samples; i++) {
|
||||
auto sample = X.index({ torch::indexing::Slice(), i });
|
||||
predictions[i] = predictSample(sample);
|
||||
if (n_classes <= STACK_THRESHOLD) {
|
||||
class_votes = stack_votes;
|
||||
std::fill_n(class_votes, n_classes, 0.0);
|
||||
} else {
|
||||
heap_votes.resize(n_classes, 0.0);
|
||||
class_votes = heap_votes.data();
|
||||
}
|
||||
|
||||
return predictions;
|
||||
double total_votes = 0.0;
|
||||
|
||||
// Optimized voting loop
|
||||
for (size_t i = 0; i < models.size(); ++i) {
|
||||
double alpha = alphas[i];
|
||||
if (alpha <= 0 || !std::isfinite(alpha)) continue;
|
||||
|
||||
int predicted_class = static_cast<DecisionTree*>(models[i].get())->predictSample(x);
|
||||
|
||||
if (__builtin_expect(predicted_class >= 0 && predicted_class < n_classes, 1)) {
|
||||
class_votes[predicted_class] += alpha;
|
||||
total_votes += alpha;
|
||||
}
|
||||
}
|
||||
|
||||
// Direct tensor creation with pre-computed size
|
||||
torch::Tensor class_probs = torch::empty({ n_classes }, torch::TensorOptions().dtype(torch::kFloat32));
|
||||
auto probs_accessor = class_probs.accessor<float, 1>();
|
||||
|
||||
if (__builtin_expect(total_votes > 0.0, 1)) {
|
||||
// Vectorized probability calculation
|
||||
const double inv_total = 1.0 / total_votes;
|
||||
for (int j = 0; j < n_classes; ++j) {
|
||||
probs_accessor[j] = static_cast<float>(class_votes[j] * inv_total);
|
||||
}
|
||||
} else {
|
||||
// Uniform distribution fallback
|
||||
const float uniform_prob = 1.0f / n_classes;
|
||||
for (int j = 0; j < n_classes; ++j) {
|
||||
probs_accessor[j] = uniform_prob;
|
||||
}
|
||||
}
|
||||
|
||||
return class_probs;
|
||||
}
|
||||
|
||||
torch::Tensor AdaBoost::predict_proba(torch::Tensor& X)
|
||||
{
|
||||
if (!fitted) {
|
||||
if (!fitted || models.empty()) {
|
||||
throw std::runtime_error(CLASSIFIER_NOT_FITTED);
|
||||
}
|
||||
|
||||
if (models.empty()) {
|
||||
throw std::runtime_error("No models have been trained");
|
||||
}
|
||||
|
||||
// X should be (n_features, n_samples)
|
||||
// Input validation
|
||||
if (X.size(0) != n) {
|
||||
throw std::runtime_error("Input has wrong number of features. Expected " +
|
||||
std::to_string(n) + " but got " + std::to_string(X.size(0)));
|
||||
}
|
||||
|
||||
int n_samples = X.size(1);
|
||||
torch::Tensor probabilities = torch::zeros({ n_samples, n_classes });
|
||||
const int n_samples = X.size(1);
|
||||
|
||||
for (int i = 0; i < n_samples; i++) {
|
||||
auto sample = X.index({ torch::indexing::Slice(), i });
|
||||
// Pre-allocate output tensor with correct layout
|
||||
torch::Tensor probabilities = torch::empty({ n_samples, n_classes },
|
||||
torch::TensorOptions().dtype(torch::kFloat32));
|
||||
|
||||
// Convert to contiguous memory if needed (optimization for memory access)
|
||||
if (!X.is_contiguous()) {
|
||||
X = X.contiguous();
|
||||
}
|
||||
|
||||
// Batch processing with memory-efficient sample extraction
|
||||
for (int i = 0; i < n_samples; ++i) {
|
||||
// Extract sample without unnecessary copies
|
||||
auto sample = X.select(1, i);
|
||||
|
||||
// Direct assignment to pre-allocated tensor
|
||||
probabilities[i] = predictProbaSample(sample);
|
||||
}
|
||||
|
||||
return probabilities;
|
||||
}
|
||||
|
||||
std::vector<int> AdaBoost::predict(std::vector<std::vector<int>>& X)
|
||||
{
|
||||
// Convert to tensor - X is samples x features, need to transpose
|
||||
torch::Tensor X_tensor = platform::TensorUtils::to_matrix(X);
|
||||
auto predictions = predict(X_tensor);
|
||||
std::vector<int> result = platform::TensorUtils::to_vector<int>(predictions);
|
||||
return result;
|
||||
}
|
||||
|
||||
std::vector<std::vector<double>> AdaBoost::predict_proba(std::vector<std::vector<int>>& X)
|
||||
{
|
||||
auto n_samples = X[0].size();
|
||||
// Convert to tensor - X is samples x features, need to transpose
|
||||
const size_t n_samples = X[0].size();
|
||||
|
||||
// Pre-allocate result with exact size
|
||||
std::vector<std::vector<double>> result;
|
||||
result.reserve(n_samples);
|
||||
|
||||
// Avoid repeated allocations
|
||||
for (size_t i = 0; i < n_samples; ++i) {
|
||||
result.emplace_back(n_classes, 0.0);
|
||||
}
|
||||
|
||||
// Convert to tensor only once (batch conversion is more efficient)
|
||||
torch::Tensor X_tensor = platform::TensorUtils::to_matrix(X);
|
||||
auto proba_tensor = predict_proba(X_tensor);
|
||||
torch::Tensor proba_tensor = predict_proba(X_tensor);
|
||||
|
||||
std::vector<std::vector<double>> result(n_samples, std::vector<double>(n_classes, 0.0));
|
||||
|
||||
for (size_t i = 0; i < n_samples; i++) {
|
||||
for (int j = 0; j < n_classes; j++) {
|
||||
result[i][j] = proba_tensor[i][j].item<double>();
|
||||
// Optimized tensor-to-vector conversion
|
||||
auto proba_accessor = proba_tensor.accessor<float, 2>();
|
||||
for (size_t i = 0; i < n_samples; ++i) {
|
||||
for (int j = 0; j < n_classes; ++j) {
|
||||
result[i][j] = static_cast<double>(proba_accessor[i][j]);
|
||||
}
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
int AdaBoost::predictSample(const torch::Tensor& x) const
|
||||
torch::Tensor AdaBoost::predict(torch::Tensor& X)
|
||||
{
|
||||
if (!fitted) {
|
||||
if (!fitted || models.empty()) {
|
||||
throw std::runtime_error(CLASSIFIER_NOT_FITTED);
|
||||
}
|
||||
|
||||
if (models.empty()) {
|
||||
throw std::runtime_error("No models have been trained");
|
||||
if (X.size(0) != n) {
|
||||
throw std::runtime_error("Input has wrong number of features. Expected " +
|
||||
std::to_string(n) + " but got " + std::to_string(X.size(0)));
|
||||
}
|
||||
|
||||
// x should be a 1D tensor with n features
|
||||
if (x.size(0) != n) {
|
||||
throw std::runtime_error("Input sample has wrong number of features. Expected " +
|
||||
std::to_string(n) + " but got " + std::to_string(x.size(0)));
|
||||
const int n_samples = X.size(1);
|
||||
|
||||
// Pre-allocate with correct dtype
|
||||
torch::Tensor predictions = torch::empty({ n_samples }, torch::TensorOptions().dtype(torch::kInt32));
|
||||
auto pred_accessor = predictions.accessor<int32_t, 1>();
|
||||
|
||||
// Ensure contiguous memory layout
|
||||
if (!X.is_contiguous()) {
|
||||
X = X.contiguous();
|
||||
}
|
||||
|
||||
// Initialize class votes
|
||||
std::vector<double> class_votes(n_classes, 0.0);
|
||||
|
||||
// Accumulate weighted votes from all estimators
|
||||
for (size_t i = 0; i < models.size(); i++) {
|
||||
if (alphas[i] <= 0) continue; // Skip estimators with zero or negative weight
|
||||
try {
|
||||
// Get prediction from this estimator
|
||||
int predicted_class = static_cast<DecisionTree*>(models[i].get())->predictSample(x);
|
||||
|
||||
// Add weighted vote for this class
|
||||
if (predicted_class >= 0 && predicted_class < n_classes) {
|
||||
class_votes[predicted_class] += alphas[i];
|
||||
}
|
||||
}
|
||||
catch (const std::exception& e) {
|
||||
std::cerr << "Error in estimator " << i << ": " << e.what() << std::endl;
|
||||
continue;
|
||||
}
|
||||
// Optimized prediction loop
|
||||
for (int i = 0; i < n_samples; ++i) {
|
||||
auto sample = X.select(1, i);
|
||||
pred_accessor[i] = predictSample(sample);
|
||||
}
|
||||
|
||||
// Return class with highest weighted vote
|
||||
return std::distance(class_votes.begin(),
|
||||
std::max_element(class_votes.begin(), class_votes.end()));
|
||||
return predictions;
|
||||
}
|
||||
|
||||
torch::Tensor AdaBoost::predictProbaSample(const torch::Tensor& x) const
|
||||
std::vector<int> AdaBoost::predict(std::vector<std::vector<int>>& X)
|
||||
{
|
||||
if (!fitted) {
|
||||
throw std::runtime_error(CLASSIFIER_NOT_FITTED);
|
||||
}
|
||||
// Single tensor conversion for batch processing
|
||||
torch::Tensor X_tensor = platform::TensorUtils::to_matrix(X);
|
||||
torch::Tensor predictions_tensor = predict(X_tensor);
|
||||
|
||||
if (models.empty()) {
|
||||
throw std::runtime_error("No models have been trained");
|
||||
}
|
||||
|
||||
// x should be a 1D tensor with n features
|
||||
if (x.size(0) != n) {
|
||||
throw std::runtime_error("Input sample has wrong number of features. Expected " +
|
||||
std::to_string(n) + " but got " + std::to_string(x.size(0)));
|
||||
}
|
||||
|
||||
// Initialize class votes (same logic as predictSample)
|
||||
std::vector<double> class_votes(n_classes, 0.0);
|
||||
|
||||
// Accumulate weighted votes from all estimators (SAMME voting)
|
||||
double total_alpha = 0.0;
|
||||
for (size_t i = 0; i < models.size(); i++) {
|
||||
if (alphas[i] <= 0) continue; // Skip estimators with zero or negative weight
|
||||
|
||||
try {
|
||||
// Get class prediction from this estimator (not probabilities!)
|
||||
int predicted_class = static_cast<DecisionTree*>(models[i].get())->predictSample(x);
|
||||
|
||||
// Add weighted vote for this class (SAMME algorithm)
|
||||
if (predicted_class >= 0 && predicted_class < n_classes) {
|
||||
class_votes[predicted_class] += alphas[i];
|
||||
total_alpha += alphas[i];
|
||||
}
|
||||
}
|
||||
catch (const std::exception& e) {
|
||||
std::cerr << "Error in estimator " << i << ": " << e.what() << std::endl;
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
// Convert votes to probabilities
|
||||
torch::Tensor class_probs = torch::zeros({ n_classes }, torch::kFloat);
|
||||
|
||||
if (total_alpha > 0) {
|
||||
// Normalize votes to get probabilities
|
||||
for (int j = 0; j < n_classes; j++) {
|
||||
class_probs[j] = static_cast<float>(class_votes[j] / total_alpha);
|
||||
}
|
||||
} else {
|
||||
// If no valid estimators, return uniform distribution
|
||||
class_probs.fill_(1.0f / n_classes);
|
||||
}
|
||||
|
||||
// Ensure probabilities are valid (they should be already, but just in case)
|
||||
class_probs = torch::clamp(class_probs, 0.0f, 1.0f);
|
||||
|
||||
// Verify they sum to 1 (they should, but normalize if needed due to floating point errors)
|
||||
float sum_probs = torch::sum(class_probs).item<float>();
|
||||
if (sum_probs > 1e-15f) {
|
||||
class_probs = class_probs / sum_probs;
|
||||
} else {
|
||||
class_probs.fill_(1.0f / n_classes);
|
||||
}
|
||||
|
||||
return class_probs;
|
||||
// Optimized tensor-to-vector conversion
|
||||
std::vector<int> result = platform::TensorUtils::to_vector<int>(predictions_tensor);
|
||||
return result;
|
||||
}
|
||||
|
||||
} // namespace bayesnet
|
@@ -14,7 +14,7 @@
|
||||
namespace bayesnet {
|
||||
class AdaBoost : public Ensemble {
|
||||
public:
|
||||
explicit AdaBoost(int n_estimators = 50, int max_depth = 1);
|
||||
explicit AdaBoost(int n_estimators = 100, int max_depth = 1);
|
||||
virtual ~AdaBoost() = default;
|
||||
|
||||
// Override base class methods
|
||||
@@ -38,7 +38,7 @@ namespace bayesnet {
|
||||
torch::Tensor predict(torch::Tensor& X) override;
|
||||
std::vector<int> predict(std::vector<std::vector<int>>& X) override;
|
||||
torch::Tensor predict_proba(torch::Tensor& X) override;
|
||||
std::vector<std::vector<double>> predict_proba(std::vector<std::vector<int>>& X);
|
||||
std::vector<std::vector<double>> predict_proba(std::vector<std::vector<int>>& X) override;
|
||||
void setDebug(bool debug) { this->debug = debug; }
|
||||
|
||||
protected:
|
||||
|
@@ -10,7 +10,7 @@
|
||||
#include <sstream>
|
||||
#include <iomanip>
|
||||
#include <limits>
|
||||
#include "TensorUtils.hpp"
|
||||
#include "common/TensorUtils.hpp"
|
||||
|
||||
namespace bayesnet {
|
||||
|
||||
|
@@ -40,7 +40,7 @@ namespace bayesnet {
|
||||
torch::Tensor predict(torch::Tensor& X) override;
|
||||
std::vector<int> predict(std::vector<std::vector<int>>& X) override;
|
||||
torch::Tensor predict_proba(torch::Tensor& X) override;
|
||||
std::vector<std::vector<double>> predict_proba(std::vector<std::vector<int>>& X);
|
||||
std::vector<std::vector<double>> predict_proba(std::vector<std::vector<int>>& X) override;
|
||||
|
||||
// Make predictions for a single sample
|
||||
int predictSample(const torch::Tensor& x) const;
|
||||
|
@@ -5,7 +5,7 @@
|
||||
// ***************************************************************
|
||||
|
||||
#include "ExpClf.h"
|
||||
#include "TensorUtils.hpp"
|
||||
#include "common/TensorUtils.hpp"
|
||||
|
||||
namespace platform {
|
||||
ExpClf::ExpClf() : semaphore_{ CountingSemaphore::getInstance() }, Boost(false)
|
||||
|
@@ -5,7 +5,7 @@
|
||||
// ***************************************************************
|
||||
|
||||
#include "ExpEnsemble.h"
|
||||
#include "TensorUtils.hpp"
|
||||
#include "common/TensorUtils.hpp"
|
||||
|
||||
namespace platform {
|
||||
ExpEnsemble::ExpEnsemble() : semaphore_{ CountingSemaphore::getInstance() }, Boost(false)
|
||||
|
@@ -5,7 +5,7 @@
|
||||
// ***************************************************************
|
||||
|
||||
#include "XA1DE.h"
|
||||
#include "TensorUtils.hpp"
|
||||
#include "common/TensorUtils.hpp"
|
||||
|
||||
namespace platform {
|
||||
void XA1DE::trainModel(const torch::Tensor& weights, const bayesnet::Smoothing_t smoothing)
|
||||
|
@@ -10,7 +10,7 @@
|
||||
#include <tuple>
|
||||
#include "XBAODE.h"
|
||||
#include "XSpode.hpp"
|
||||
#include "TensorUtils.hpp"
|
||||
#include "common/TensorUtils.hpp"
|
||||
#include <loguru.hpp>
|
||||
|
||||
namespace platform {
|
||||
|
@@ -246,8 +246,8 @@ namespace platform {
|
||||
//
|
||||
clf->fit(X_train, y_train, features, className, states, smooth_type);
|
||||
auto clf_notes = clf->getNotes();
|
||||
std::transform(clf_notes.begin(), clf_notes.end(), std::back_inserter(notes), [nfold](const std::string& note)
|
||||
{ return "Fold " + std::to_string(nfold) + ": " + note; });
|
||||
std::transform(clf_notes.begin(), clf_notes.end(), std::back_inserter(notes), [seed, nfold](const std::string& note)
|
||||
{ return "Seed: " + std::to_string(seed) + " Fold: " + std::to_string(nfold) + ": " + note; });
|
||||
nodes[item] = clf->getNumberOfNodes();
|
||||
edges[item] = clf->getNumberOfEdges();
|
||||
num_states[item] = clf->getNumberOfStates();
|
||||
|
@@ -3,6 +3,7 @@
|
||||
#include <numeric>
|
||||
#include <utility>
|
||||
#include "RocAuc.h"
|
||||
#include "common/TensorUtils.hpp" // tensorToVector
|
||||
namespace platform {
|
||||
|
||||
double RocAuc::compute(const torch::Tensor& y_proba, const torch::Tensor& labels)
|
||||
|
@@ -1,6 +1,6 @@
|
||||
#include <sstream>
|
||||
#include "Scores.h"
|
||||
#include "common/Utils.h" // tensorToVector
|
||||
#include "common/TensorUtils.hpp" // tensorToVector
|
||||
#include "common/Colors.h"
|
||||
namespace platform {
|
||||
Scores::Scores(torch::Tensor& y_test, torch::Tensor& y_proba, int num_classes, std::vector<std::string> labels) : num_classes(num_classes), labels(labels), y_test(y_test), y_proba(y_proba)
|
||||
@@ -50,7 +50,7 @@ namespace platform {
|
||||
auto nClasses = num_classes;
|
||||
if (num_classes == 2)
|
||||
nClasses = 1;
|
||||
auto y_testv = tensorToVector<int>(y_test);
|
||||
auto y_testv = TensorUtils::tensorToVector<int>(y_test);
|
||||
std::vector<double> aucScores(nClasses, 0.0);
|
||||
std::vector<std::pair<double, int>> scoresAndLabels;
|
||||
for (size_t classIdx = 0; classIdx < nClasses; ++classIdx) {
|
||||
|
@@ -54,10 +54,8 @@ namespace platform {
|
||||
}
|
||||
void ExcelFile::setProperties(std::string title)
|
||||
{
|
||||
char line[title.size() + 1];
|
||||
strcpy(line, title.c_str());
|
||||
lxw_doc_properties properties = {
|
||||
.title = line,
|
||||
.title = title.c_str(),
|
||||
.subject = (char*)"Machine learning results",
|
||||
.author = (char*)"Ricardo Montañana Gómez",
|
||||
.manager = (char*)"Dr. J. A. Gámez, Dr. J. M. Puerta",
|
||||
|
@@ -2,14 +2,7 @@ if(ENABLE_TESTING)
|
||||
set(TEST_PLATFORM "unit_tests_platform")
|
||||
include_directories(
|
||||
${Platform_SOURCE_DIR}/src
|
||||
${Platform_SOURCE_DIR}/lib/argparse/include
|
||||
${Platform_SOURCE_DIR}/lib/mdlp/src
|
||||
${Platform_SOURCE_DIR}/lib/Files
|
||||
${Platform_SOURCE_DIR}/lib/json/include
|
||||
${Platform_SOURCE_DIR}/lib/folding
|
||||
${CMAKE_BINARY_DIR}/configured_files/include
|
||||
${PyClassifiers_INCLUDE_DIRS}
|
||||
${Bayesnet_INCLUDE_DIRS}
|
||||
)
|
||||
set(TEST_SOURCES_PLATFORM
|
||||
TestUtils.cpp TestPlatform.cpp TestResult.cpp TestScores.cpp TestDecisionTree.cpp TestAdaBoost.cpp
|
||||
@@ -19,6 +12,7 @@ if(ENABLE_TESTING)
|
||||
${Platform_SOURCE_DIR}/src/experimental_clfs/AdaBoost.cpp
|
||||
)
|
||||
add_executable(${TEST_PLATFORM} ${TEST_SOURCES_PLATFORM})
|
||||
target_link_libraries(${TEST_PLATFORM} PUBLIC "${TORCH_LIBRARIES}" fimdlp Catch2::Catch2WithMain bayesnet)
|
||||
target_link_libraries(${TEST_PLATFORM} PUBLIC
|
||||
torch::torch fimdlp:fimdlp Catch2::Catch2WithMain bayesnet::bayesnet pyclassifiers::pyclassifiers)
|
||||
add_test(NAME ${TEST_PLATFORM} COMMAND ${TEST_PLATFORM})
|
||||
endif(ENABLE_TESTING)
|
||||
|
@@ -13,12 +13,13 @@
|
||||
#include <stdexcept>
|
||||
#include "experimental_clfs/AdaBoost.h"
|
||||
#include "experimental_clfs/DecisionTree.h"
|
||||
#include "experimental_clfs/TensorUtils.hpp"
|
||||
#include "common/TensorUtils.hpp"
|
||||
#include "TestUtils.h"
|
||||
|
||||
using namespace bayesnet;
|
||||
using namespace Catch::Matchers;
|
||||
|
||||
static const bool DEBUG = false;
|
||||
|
||||
TEST_CASE("AdaBoost Construction", "[AdaBoost]")
|
||||
{
|
||||
@@ -141,6 +142,7 @@ TEST_CASE("AdaBoost Basic Functionality", "[AdaBoost]")
|
||||
SECTION("Prediction with vector interface")
|
||||
{
|
||||
AdaBoost ada(10, 3);
|
||||
ada.setDebug(DEBUG); // Enable debug to investigate
|
||||
ada.fit(X, y, features, className, states, Smoothing_t::NONE);
|
||||
|
||||
auto predictions = ada.predict(X);
|
||||
@@ -159,6 +161,7 @@ TEST_CASE("AdaBoost Basic Functionality", "[AdaBoost]")
|
||||
SECTION("Probability predictions with vector interface")
|
||||
{
|
||||
AdaBoost ada(10, 3);
|
||||
ada.setDebug(DEBUG); // ENABLE DEBUG HERE TOO
|
||||
ada.fit(X, y, features, className, states, Smoothing_t::NONE);
|
||||
|
||||
auto proba = ada.predict_proba(X);
|
||||
@@ -183,8 +186,16 @@ TEST_CASE("AdaBoost Basic Functionality", "[AdaBoost]")
|
||||
correct++;
|
||||
}
|
||||
|
||||
// Check that predict_proba matches the expected predict value
|
||||
REQUIRE(pred == (p[0] > p[1] ? 0 : 1));
|
||||
INFO("Probability test - Sample " << i << ": pred=" << pred << ", probs=[" << p[0] << "," << p[1] << "], expected_from_probs=" << predicted_class);
|
||||
|
||||
// Handle ties
|
||||
if (std::abs(p[0] - p[1]) < 1e-10) {
|
||||
INFO("Tie detected in probabilities");
|
||||
// Either prediction is valid in case of tie
|
||||
} else {
|
||||
// Check that predict_proba matches the expected predict value
|
||||
REQUIRE(pred == predicted_class);
|
||||
}
|
||||
}
|
||||
double accuracy = static_cast<double>(correct) / n_samples;
|
||||
REQUIRE(accuracy > 0.99); // Should achieve good accuracy on this simple dataset
|
||||
@@ -230,103 +241,50 @@ TEST_CASE("AdaBoost Tensor Interface", "[AdaBoost]")
|
||||
}
|
||||
}
|
||||
|
||||
TEST_CASE("AdaBoost on Iris Dataset", "[AdaBoost][iris]")
|
||||
TEST_CASE("AdaBoost SAMME Algorithm Validation", "[AdaBoost]")
|
||||
{
|
||||
auto raw = RawDatasets("iris", true);
|
||||
|
||||
SECTION("Training with vector interface")
|
||||
SECTION("Prediction consistency with probabilities")
|
||||
{
|
||||
AdaBoost ada(30, 3);
|
||||
|
||||
REQUIRE_NOTHROW(ada.fit(raw.Xv, raw.yv, raw.featuresv, raw.classNamev, raw.statesv, Smoothing_t::NONE));
|
||||
|
||||
auto predictions = ada.predict(raw.Xv);
|
||||
REQUIRE(predictions.size() == raw.yv.size());
|
||||
|
||||
// Calculate accuracy
|
||||
int correct = 0;
|
||||
for (size_t i = 0; i < predictions.size(); i++) {
|
||||
if (predictions[i] == raw.yv[i]) correct++;
|
||||
}
|
||||
double accuracy = static_cast<double>(correct) / raw.yv.size();
|
||||
REQUIRE(accuracy > 0.85); // Should achieve good accuracy
|
||||
|
||||
// Test probability predictions
|
||||
auto proba = ada.predict_proba(raw.Xv);
|
||||
REQUIRE(proba.size() == raw.yv.size());
|
||||
REQUIRE(proba[0].size() == 3); // Three classes
|
||||
|
||||
// Verify estimator weights and errors
|
||||
auto weights = ada.getEstimatorWeights();
|
||||
auto errors = ada.getTrainingErrors();
|
||||
|
||||
REQUIRE(weights.size() == errors.size());
|
||||
REQUIRE(weights.size() > 0);
|
||||
|
||||
// All weights should be positive (for non-zero error estimators)
|
||||
for (double w : weights) {
|
||||
REQUIRE(w >= 0.0);
|
||||
}
|
||||
|
||||
// All errors should be less than 0.5 (better than random)
|
||||
for (double e : errors) {
|
||||
REQUIRE(e < 0.5);
|
||||
REQUIRE(e >= 0.0);
|
||||
}
|
||||
}
|
||||
|
||||
SECTION("Different number of estimators")
|
||||
{
|
||||
std::vector<int> n_estimators = { 5, 15, 25 };
|
||||
|
||||
for (int n_est : n_estimators) {
|
||||
AdaBoost ada(n_est, 2);
|
||||
ada.fit(raw.dataset, raw.featurest, raw.classNamet, raw.statest, Smoothing_t::NONE);
|
||||
|
||||
auto predictions = ada.predict(raw.Xt);
|
||||
REQUIRE(predictions.size(0) == raw.yt.size(0));
|
||||
|
||||
// Check that we don't exceed the specified number of estimators
|
||||
auto weights = ada.getEstimatorWeights();
|
||||
REQUIRE(static_cast<int>(weights.size()) <= n_est);
|
||||
}
|
||||
}
|
||||
|
||||
SECTION("Different base estimator depths")
|
||||
{
|
||||
std::vector<int> depths = { 1, 2, 4 };
|
||||
|
||||
for (int depth : depths) {
|
||||
AdaBoost ada(15, depth);
|
||||
ada.fit(raw.dataset, raw.featurest, raw.classNamet, raw.statest, Smoothing_t::NONE);
|
||||
|
||||
auto predictions = ada.predict(raw.Xt);
|
||||
REQUIRE(predictions.size(0) == raw.yt.size(0));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
TEST_CASE("AdaBoost Edge Cases", "[AdaBoost]")
|
||||
{
|
||||
auto raw = RawDatasets("iris", true);
|
||||
|
||||
SECTION("Single estimator (depth 1 stump)")
|
||||
{
|
||||
AdaBoost ada(1, 1); // Single decision stump
|
||||
AdaBoost ada(15, 3);
|
||||
ada.setDebug(DEBUG); // Enable debug for ALL instances
|
||||
ada.fit(raw.dataset, raw.featurest, raw.classNamet, raw.statest, Smoothing_t::NONE);
|
||||
|
||||
auto predictions = ada.predict(raw.Xt);
|
||||
REQUIRE(predictions.size(0) == raw.yt.size(0));
|
||||
auto probabilities = ada.predict_proba(raw.Xt);
|
||||
|
||||
auto weights = ada.getEstimatorWeights();
|
||||
REQUIRE(weights.size() == 1);
|
||||
REQUIRE(predictions.size(0) == probabilities.size(0));
|
||||
REQUIRE(probabilities.size(1) == 3); // Three classes in Iris
|
||||
|
||||
// For each sample, predicted class should correspond to highest probability
|
||||
for (int i = 0; i < predictions.size(0); i++) {
|
||||
int predicted_class = predictions[i].item<int>();
|
||||
auto probs = probabilities[i];
|
||||
|
||||
// Find class with highest probability
|
||||
auto max_prob_idx = torch::argmax(probs).item<int>();
|
||||
|
||||
// Predicted class should match class with highest probability
|
||||
REQUIRE(predicted_class == max_prob_idx);
|
||||
|
||||
// Probabilities should sum to 1
|
||||
double sum_probs = torch::sum(probs).item<double>();
|
||||
REQUIRE(sum_probs == Catch::Approx(1.0).epsilon(1e-6));
|
||||
|
||||
// All probabilities should be non-negative
|
||||
for (int j = 0; j < 3; j++) {
|
||||
REQUIRE(probs[j].item<double>() >= 0.0);
|
||||
REQUIRE(probs[j].item<double>() <= 1.0);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
SECTION("Perfect classifier scenario")
|
||||
SECTION("Weighted voting verification")
|
||||
{
|
||||
// Create a perfectly separable dataset
|
||||
// Simple dataset where we can verify the weighted voting
|
||||
std::vector<std::vector<int>> X = { {0,0,1,1}, {0,1,0,1} };
|
||||
std::vector<int> y = { 0, 0, 1, 1 };
|
||||
std::vector<int> y = { 0, 1, 1, 0 };
|
||||
std::vector<std::string> features = { "f1", "f2" };
|
||||
std::string className = "class";
|
||||
std::map<std::string, std::vector<int>> states;
|
||||
@@ -334,191 +292,61 @@ TEST_CASE("AdaBoost Edge Cases", "[AdaBoost]")
|
||||
states["f2"] = { 0, 1 };
|
||||
states["class"] = { 0, 1 };
|
||||
|
||||
AdaBoost ada(10, 3);
|
||||
ada.fit(X, y, features, className, states, Smoothing_t::NONE);
|
||||
|
||||
auto predictions = ada.predict(X);
|
||||
REQUIRE(predictions.size() == 4);
|
||||
|
||||
// Should achieve perfect accuracy
|
||||
int correct = 0;
|
||||
for (size_t i = 0; i < predictions.size(); i++) {
|
||||
if (predictions[i] == y[i]) correct++;
|
||||
}
|
||||
REQUIRE(correct == 4);
|
||||
|
||||
// Should stop early due to perfect classification
|
||||
auto errors = ada.getTrainingErrors();
|
||||
if (errors.size() > 0) {
|
||||
REQUIRE(errors.back() < 1e-10); // Very low error
|
||||
}
|
||||
}
|
||||
|
||||
SECTION("Small dataset")
|
||||
{
|
||||
// Very small dataset
|
||||
std::vector<std::vector<int>> X = { {0,1}, {1,0} };
|
||||
std::vector<int> y = { 0, 1 };
|
||||
std::vector<std::string> features = { "f1", "f2" };
|
||||
std::string className = "class";
|
||||
std::map<std::string, std::vector<int>> states;
|
||||
states["f1"] = { 0, 1 };
|
||||
states["f2"] = { 0, 1 };
|
||||
states["class"] = { 0, 1 };
|
||||
|
||||
AdaBoost ada(5, 1);
|
||||
REQUIRE_NOTHROW(ada.fit(X, y, features, className, states, Smoothing_t::NONE));
|
||||
|
||||
auto predictions = ada.predict(X);
|
||||
REQUIRE(predictions.size() == 2);
|
||||
}
|
||||
}
|
||||
|
||||
TEST_CASE("AdaBoost Graph Visualization", "[AdaBoost]")
|
||||
{
|
||||
// Simple dataset for visualization
|
||||
std::vector<std::vector<int>> X = { {0,0,1,1}, {0,1,0,1} };
|
||||
std::vector<int> y = { 0, 1, 1, 0 }; // XOR pattern
|
||||
std::vector<std::string> features = { "x1", "x2" };
|
||||
std::string className = "xor";
|
||||
std::map<std::string, std::vector<int>> states;
|
||||
states["x1"] = { 0, 1 };
|
||||
states["x2"] = { 0, 1 };
|
||||
states["xor"] = { 0, 1 };
|
||||
|
||||
SECTION("Graph generation")
|
||||
{
|
||||
AdaBoost ada(5, 2);
|
||||
ada.setDebug(DEBUG); // Enable debug for detailed logging
|
||||
ada.fit(X, y, features, className, states, Smoothing_t::NONE);
|
||||
|
||||
auto graph_lines = ada.graph();
|
||||
INFO("=== Final test verification ===");
|
||||
auto predictions = ada.predict(X);
|
||||
auto probabilities = ada.predict_proba(X);
|
||||
auto alphas = ada.getEstimatorWeights();
|
||||
|
||||
REQUIRE(graph_lines.size() > 2);
|
||||
REQUIRE(graph_lines.front() == "digraph AdaBoost {");
|
||||
REQUIRE(graph_lines.back() == "}");
|
||||
|
||||
// Should contain base estimator references
|
||||
bool has_estimators = false;
|
||||
for (const auto& line : graph_lines) {
|
||||
if (line.find("Estimator") != std::string::npos) {
|
||||
has_estimators = true;
|
||||
break;
|
||||
}
|
||||
INFO("Training info:");
|
||||
for (size_t i = 0; i < alphas.size(); i++) {
|
||||
INFO(" Model " << i << ": alpha=" << alphas[i]);
|
||||
}
|
||||
REQUIRE(has_estimators);
|
||||
|
||||
// Should contain alpha values
|
||||
bool has_alpha = false;
|
||||
for (const auto& line : graph_lines) {
|
||||
if (line.find("α") != std::string::npos || line.find("alpha") != std::string::npos) {
|
||||
has_alpha = true;
|
||||
break;
|
||||
}
|
||||
REQUIRE(predictions.size() == 4);
|
||||
REQUIRE(probabilities.size() == 4);
|
||||
REQUIRE(probabilities[0].size() == 2); // Two classes
|
||||
REQUIRE(alphas.size() > 0);
|
||||
|
||||
// Verify that estimator weights are reasonable
|
||||
for (double alpha : alphas) {
|
||||
REQUIRE(alpha >= 0.0); // Alphas should be non-negative
|
||||
}
|
||||
REQUIRE(has_alpha);
|
||||
}
|
||||
|
||||
SECTION("Graph with title")
|
||||
{
|
||||
AdaBoost ada(3, 1);
|
||||
ada.fit(X, y, features, className, states, Smoothing_t::NONE);
|
||||
// Verify prediction-probability consistency with detailed logging
|
||||
for (size_t i = 0; i < predictions.size(); i++) {
|
||||
int pred = predictions[i];
|
||||
auto probs = probabilities[i];
|
||||
|
||||
auto graph_lines = ada.graph("XOR AdaBoost");
|
||||
INFO("Final check - Sample " << i << ": predicted=" << pred << ", probabilities=[" << probs[0] << "," << probs[1] << "]");
|
||||
|
||||
bool has_title = false;
|
||||
for (const auto& line : graph_lines) {
|
||||
if (line.find("label=\"XOR AdaBoost\"") != std::string::npos) {
|
||||
has_title = true;
|
||||
break;
|
||||
// Handle the case where probabilities are exactly equal (tie)
|
||||
if (std::abs(probs[0] - probs[1]) < 1e-10) {
|
||||
INFO("Tie detected in probabilities - either prediction is valid");
|
||||
REQUIRE((pred == 0 || pred == 1));
|
||||
} else {
|
||||
// Normal case - prediction should match max probability
|
||||
int expected_pred = (probs[0] > probs[1]) ? 0 : 1;
|
||||
INFO("Expected prediction based on probs: " << expected_pred);
|
||||
REQUIRE(pred == expected_pred);
|
||||
}
|
||||
|
||||
REQUIRE(probs[0] + probs[1] == Catch::Approx(1.0).epsilon(1e-6));
|
||||
}
|
||||
REQUIRE(has_title);
|
||||
}
|
||||
}
|
||||
|
||||
TEST_CASE("AdaBoost with Weights", "[AdaBoost]")
|
||||
{
|
||||
auto raw = RawDatasets("iris", true);
|
||||
|
||||
SECTION("Uniform weights")
|
||||
{
|
||||
AdaBoost ada(20, 3);
|
||||
ada.fit(raw.dataset, raw.featurest, raw.classNamet, raw.statest, raw.weights, Smoothing_t::NONE);
|
||||
|
||||
auto predictions = ada.predict(raw.Xt);
|
||||
REQUIRE(predictions.size(0) == raw.yt.size(0));
|
||||
|
||||
auto weights = ada.getEstimatorWeights();
|
||||
REQUIRE(weights.size() > 0);
|
||||
}
|
||||
|
||||
SECTION("Non-uniform weights")
|
||||
SECTION("Empty models edge case")
|
||||
{
|
||||
auto weights = torch::ones({ raw.nSamples });
|
||||
weights.index({ torch::indexing::Slice(0, 50) }) *= 3.0; // Emphasize first class
|
||||
weights = weights / weights.sum();
|
||||
AdaBoost ada(1, 1);
|
||||
ada.setDebug(DEBUG); // Enable debug for ALL instances
|
||||
|
||||
AdaBoost ada(15, 2);
|
||||
ada.fit(raw.dataset, raw.featurest, raw.classNamet, raw.statest, weights, Smoothing_t::NONE);
|
||||
|
||||
auto predictions = ada.predict(raw.Xt);
|
||||
REQUIRE(predictions.size(0) == raw.yt.size(0));
|
||||
|
||||
// Check that training completed successfully
|
||||
auto estimator_weights = ada.getEstimatorWeights();
|
||||
auto errors = ada.getTrainingErrors();
|
||||
|
||||
REQUIRE(estimator_weights.size() == errors.size());
|
||||
REQUIRE(estimator_weights.size() > 0);
|
||||
}
|
||||
}
|
||||
|
||||
TEST_CASE("AdaBoost Input Dimension Validation", "[AdaBoost]")
|
||||
{
|
||||
auto raw = RawDatasets("iris", true);
|
||||
|
||||
SECTION("Correct input dimensions")
|
||||
{
|
||||
AdaBoost ada(10, 2);
|
||||
ada.fit(raw.dataset, raw.featurest, raw.classNamet, raw.statest, Smoothing_t::NONE);
|
||||
|
||||
// Test with correct tensor dimensions (features x samples)
|
||||
REQUIRE_NOTHROW(ada.predict(raw.Xt));
|
||||
REQUIRE_NOTHROW(ada.predict_proba(raw.Xt));
|
||||
|
||||
// Test with correct vector dimensions (features x samples)
|
||||
REQUIRE_NOTHROW(ada.predict(raw.Xv));
|
||||
REQUIRE_NOTHROW(ada.predict_proba(raw.Xv));
|
||||
}
|
||||
|
||||
SECTION("Dimension consistency between interfaces")
|
||||
{
|
||||
AdaBoost ada(10, 2);
|
||||
ada.fit(raw.dataset, raw.featurest, raw.classNamet, raw.statest, Smoothing_t::NONE);
|
||||
|
||||
// Get predictions from both interfaces
|
||||
auto tensor_predictions = ada.predict(raw.Xt);
|
||||
auto vector_predictions = ada.predict(raw.Xv);
|
||||
|
||||
// Should have same number of predictions
|
||||
REQUIRE(tensor_predictions.size(0) == static_cast<int>(vector_predictions.size()));
|
||||
|
||||
// Test probability predictions
|
||||
auto tensor_proba = ada.predict_proba(raw.Xt);
|
||||
auto vector_proba = ada.predict_proba(raw.Xv);
|
||||
|
||||
REQUIRE(tensor_proba.size(0) == static_cast<int>(vector_proba.size()));
|
||||
REQUIRE(tensor_proba.size(1) == static_cast<int>(vector_proba[0].size()));
|
||||
|
||||
// Verify predictions match between interfaces
|
||||
for (int i = 0; i < tensor_predictions.size(0); i++) {
|
||||
REQUIRE(tensor_predictions[i].item<int>() == vector_predictions[i]);
|
||||
|
||||
// Verify probabilities match between interfaces
|
||||
for (int j = 0; j < tensor_proba.size(1); j++) {
|
||||
REQUIRE(tensor_proba[i][j].item<double>() == Catch::Approx(vector_proba[i][j]).epsilon(1e-10));
|
||||
}
|
||||
}
|
||||
// Try to predict before fitting
|
||||
std::vector<std::vector<int>> X = { {0}, {1} };
|
||||
REQUIRE_THROWS_WITH(ada.predict(X), ContainsSubstring("not been fitted"));
|
||||
REQUIRE_THROWS_WITH(ada.predict_proba(X), ContainsSubstring("not been fitted"));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -548,6 +376,7 @@ TEST_CASE("AdaBoost Debug - Simple Dataset Analysis", "[AdaBoost][debug]")
|
||||
SECTION("Debug training process")
|
||||
{
|
||||
AdaBoost ada(5, 3); // Few estimators for debugging
|
||||
ada.setDebug(DEBUG);
|
||||
|
||||
// This should work perfectly on this simple dataset
|
||||
REQUIRE_NOTHROW(ada.fit(X, y, features, className, states, Smoothing_t::NONE));
|
||||
@@ -603,7 +432,14 @@ TEST_CASE("AdaBoost Debug - Simple Dataset Analysis", "[AdaBoost][debug]")
|
||||
|
||||
// Predicted class should match highest probability
|
||||
int pred_class = predictions[i];
|
||||
REQUIRE(pred_class == (p[0] > p[1] ? 0 : 1));
|
||||
|
||||
// Handle ties
|
||||
if (std::abs(p[0] - p[1]) < 1e-10) {
|
||||
INFO("Tie detected - probabilities are equal");
|
||||
REQUIRE((pred_class == 0 || pred_class == 1));
|
||||
} else {
|
||||
REQUIRE(pred_class == (p[0] > p[1] ? 0 : 1));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -621,6 +457,7 @@ TEST_CASE("AdaBoost Debug - Simple Dataset Analysis", "[AdaBoost][debug]")
|
||||
double tree_accuracy = static_cast<double>(tree_correct) / n_samples;
|
||||
|
||||
AdaBoost ada(5, 3);
|
||||
ada.setDebug(DEBUG);
|
||||
ada.fit(X, y, features, className, states, Smoothing_t::NONE);
|
||||
auto ada_predictions = ada.predict(X);
|
||||
|
||||
@@ -639,95 +476,6 @@ TEST_CASE("AdaBoost Debug - Simple Dataset Analysis", "[AdaBoost][debug]")
|
||||
}
|
||||
}
|
||||
|
||||
TEST_CASE("AdaBoost SAMME Algorithm Validation", "[AdaBoost]")
|
||||
{
|
||||
auto raw = RawDatasets("iris", true);
|
||||
|
||||
SECTION("Prediction consistency with probabilities")
|
||||
{
|
||||
AdaBoost ada(15, 3);
|
||||
ada.fit(raw.dataset, raw.featurest, raw.classNamet, raw.statest, Smoothing_t::NONE);
|
||||
|
||||
auto predictions = ada.predict(raw.Xt);
|
||||
auto probabilities = ada.predict_proba(raw.Xt);
|
||||
|
||||
REQUIRE(predictions.size(0) == probabilities.size(0));
|
||||
REQUIRE(probabilities.size(1) == 3); // Three classes in Iris
|
||||
|
||||
// For each sample, predicted class should correspond to highest probability
|
||||
for (int i = 0; i < predictions.size(0); i++) {
|
||||
int predicted_class = predictions[i].item<int>();
|
||||
auto probs = probabilities[i];
|
||||
|
||||
// Find class with highest probability
|
||||
auto max_prob_idx = torch::argmax(probs).item<int>();
|
||||
|
||||
// Predicted class should match class with highest probability
|
||||
REQUIRE(predicted_class == max_prob_idx);
|
||||
|
||||
// Probabilities should sum to 1
|
||||
double sum_probs = torch::sum(probs).item<double>();
|
||||
REQUIRE(sum_probs == Catch::Approx(1.0).epsilon(1e-6));
|
||||
|
||||
// All probabilities should be non-negative
|
||||
for (int j = 0; j < 3; j++) {
|
||||
REQUIRE(probs[j].item<double>() >= 0.0);
|
||||
REQUIRE(probs[j].item<double>() <= 1.0);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
SECTION("Weighted voting verification")
|
||||
{
|
||||
// Simple dataset where we can verify the weighted voting
|
||||
std::vector<std::vector<int>> X = { {0,0,1,1}, {0,1,0,1} };
|
||||
std::vector<int> y = { 0, 1, 1, 0 };
|
||||
std::vector<std::string> features = { "f1", "f2" };
|
||||
std::string className = "class";
|
||||
std::map<std::string, std::vector<int>> states;
|
||||
states["f1"] = { 0, 1 };
|
||||
states["f2"] = { 0, 1 };
|
||||
states["class"] = { 0, 1 };
|
||||
|
||||
AdaBoost ada(5, 2);
|
||||
ada.fit(X, y, features, className, states, Smoothing_t::NONE);
|
||||
|
||||
auto predictions = ada.predict(X);
|
||||
auto probabilities = ada.predict_proba(X);
|
||||
auto alphas = ada.getEstimatorWeights();
|
||||
|
||||
REQUIRE(predictions.size() == 4);
|
||||
REQUIRE(probabilities.size() == 4);
|
||||
REQUIRE(probabilities[0].size() == 2); // Two classes
|
||||
REQUIRE(alphas.size() > 0);
|
||||
|
||||
// Verify that estimator weights are reasonable
|
||||
for (double alpha : alphas) {
|
||||
REQUIRE(alpha >= 0.0); // Alphas should be non-negative
|
||||
}
|
||||
|
||||
// Verify prediction-probability consistency
|
||||
for (size_t i = 0; i < predictions.size(); i++) {
|
||||
int pred = predictions[i];
|
||||
auto probs = probabilities[i];
|
||||
INFO("Sample " << i << ": predicted=" << pred
|
||||
<< ", probabilities=[" << probs[0] << ", " << probs[1] << "]");
|
||||
|
||||
REQUIRE(pred == (probs[0] > probs[1] ? 0 : 1));
|
||||
REQUIRE(probs[0] + probs[1] == Catch::Approx(1.0).epsilon(1e-6));
|
||||
}
|
||||
}
|
||||
|
||||
SECTION("Empty models edge case")
|
||||
{
|
||||
AdaBoost ada(1, 1);
|
||||
|
||||
// Try to predict before fitting
|
||||
std::vector<std::vector<int>> X = { {0}, {1} };
|
||||
REQUIRE_THROWS_WITH(ada.predict(X), ContainsSubstring("not been fitted"));
|
||||
REQUIRE_THROWS_WITH(ada.predict_proba(X), ContainsSubstring("not been fitted"));
|
||||
}
|
||||
}
|
||||
TEST_CASE("AdaBoost Predict-Proba Consistency Fix", "[AdaBoost][consistency]")
|
||||
{
|
||||
// Simple binary classification dataset
|
||||
@@ -743,20 +491,31 @@ TEST_CASE("AdaBoost Predict-Proba Consistency Fix", "[AdaBoost][consistency]")
|
||||
SECTION("Binary classification consistency")
|
||||
{
|
||||
AdaBoost ada(3, 2);
|
||||
ada.setDebug(true); // Enable debug output
|
||||
ada.setDebug(DEBUG); // Enable debug output
|
||||
ada.fit(X, y, features, className, states, Smoothing_t::NONE);
|
||||
|
||||
INFO("=== Debugging predict vs predict_proba consistency ===");
|
||||
|
||||
// Get training info
|
||||
auto alphas = ada.getEstimatorWeights();
|
||||
auto errors = ada.getTrainingErrors();
|
||||
|
||||
INFO("Training completed:");
|
||||
INFO(" Number of models: " << alphas.size());
|
||||
for (size_t i = 0; i < alphas.size(); i++) {
|
||||
INFO(" Model " << i << ": alpha=" << alphas[i] << ", error=" << errors[i]);
|
||||
}
|
||||
|
||||
auto predictions = ada.predict(X);
|
||||
auto probabilities = ada.predict_proba(X);
|
||||
|
||||
INFO("=== Debugging predict vs predict_proba consistency ===");
|
||||
|
||||
// Verify consistency for each sample
|
||||
for (size_t i = 0; i < predictions.size(); i++) {
|
||||
int predicted_class = predictions[i];
|
||||
auto probs = probabilities[i];
|
||||
|
||||
INFO("Sample " << i << ":");
|
||||
INFO(" Features: [" << X[0][i] << ", " << X[1][i] << "]");
|
||||
INFO(" True class: " << y[i]);
|
||||
INFO(" Predicted class: " << predicted_class);
|
||||
INFO(" Probabilities: [" << probs[0] << ", " << probs[1] << "]");
|
||||
@@ -765,7 +524,14 @@ TEST_CASE("AdaBoost Predict-Proba Consistency Fix", "[AdaBoost][consistency]")
|
||||
int max_prob_class = (probs[0] > probs[1]) ? 0 : 1;
|
||||
INFO(" Max prob class: " << max_prob_class);
|
||||
|
||||
REQUIRE(predicted_class == max_prob_class);
|
||||
// Handle tie case (when probabilities are equal)
|
||||
if (std::abs(probs[0] - probs[1]) < 1e-10) {
|
||||
INFO(" Tie detected - probabilities are equal");
|
||||
// In case of tie, either prediction is valid
|
||||
REQUIRE((predicted_class == 0 || predicted_class == 1));
|
||||
} else {
|
||||
REQUIRE(predicted_class == max_prob_class);
|
||||
}
|
||||
|
||||
// Probabilities should sum to 1
|
||||
double sum_probs = probs[0] + probs[1];
|
||||
@@ -778,37 +544,4 @@ TEST_CASE("AdaBoost Predict-Proba Consistency Fix", "[AdaBoost][consistency]")
|
||||
REQUIRE(probs[1] <= 1.0);
|
||||
}
|
||||
}
|
||||
|
||||
SECTION("Multi-class consistency")
|
||||
{
|
||||
auto raw = RawDatasets("iris", true);
|
||||
|
||||
AdaBoost ada(5, 2);
|
||||
ada.fit(raw.dataset, raw.featurest, raw.classNamet, raw.statest, Smoothing_t::NONE);
|
||||
|
||||
auto predictions = ada.predict(raw.Xt);
|
||||
auto probabilities = ada.predict_proba(raw.Xt);
|
||||
|
||||
// Check consistency for first 10 samples
|
||||
for (int i = 0; i < std::min(static_cast<int64_t>(10), predictions.size(0)); i++) {
|
||||
int predicted_class = predictions[i].item<int>();
|
||||
auto probs = probabilities[i];
|
||||
|
||||
// Find class with maximum probability
|
||||
auto max_prob_idx = torch::argmax(probs).item<int>();
|
||||
|
||||
INFO("Sample " << i << ":");
|
||||
INFO(" Predicted class: " << predicted_class);
|
||||
INFO(" Max prob class: " << max_prob_idx);
|
||||
INFO(" Probabilities: [" << probs[0].item<float>() << ", "
|
||||
<< probs[1].item<float>() << ", " << probs[2].item<float>() << "]");
|
||||
|
||||
// They must match
|
||||
REQUIRE(predicted_class == max_prob_idx);
|
||||
|
||||
// Probabilities should sum to 1
|
||||
double sum_probs = torch::sum(probs).item<double>();
|
||||
REQUIRE(sum_probs == Catch::Approx(1.0).epsilon(1e-6));
|
||||
}
|
||||
}
|
||||
}
|
@@ -7,7 +7,7 @@
|
||||
#include <string>
|
||||
#include "TestUtils.h"
|
||||
#include "folding.hpp"
|
||||
#include <ArffFiles/ArffFiles.hpp>
|
||||
#include <ArffFiles.hpp>
|
||||
#include <bayesnet/classifiers/TAN.h>
|
||||
#include "config_platform.h"
|
||||
|
||||
|
@@ -1,21 +0,0 @@
|
||||
{
|
||||
"default-registry": {
|
||||
"kind": "git",
|
||||
"baseline": "760bfd0c8d7c89ec640aec4df89418b7c2745605",
|
||||
"repository": "https://github.com/microsoft/vcpkg"
|
||||
},
|
||||
"registries": [
|
||||
{
|
||||
"kind": "git",
|
||||
"repository": "https://github.com/rmontanana/vcpkg-stash",
|
||||
"baseline": "1ea69243c0e8b0de77c9d1dd6e1d7593ae7f3627",
|
||||
"packages": [
|
||||
"arff-files",
|
||||
"bayesnet",
|
||||
"fimdlp",
|
||||
"folding",
|
||||
"libtorch-bin"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
43
vcpkg.json
43
vcpkg.json
@@ -1,43 +0,0 @@
|
||||
{
|
||||
"name": "platform",
|
||||
"version-string": "1.1.0",
|
||||
"dependencies": [
|
||||
"arff-files",
|
||||
"nlohmann-json",
|
||||
"fimdlp",
|
||||
"libtorch-bin",
|
||||
"folding",
|
||||
"catch2",
|
||||
"argparse"
|
||||
],
|
||||
"overrides": [
|
||||
{
|
||||
"name": "arff-files",
|
||||
"version": "1.1.0"
|
||||
},
|
||||
{
|
||||
"name": "fimdlp",
|
||||
"version": "2.0.1"
|
||||
},
|
||||
{
|
||||
"name": "libtorch-bin",
|
||||
"version": "2.7.0"
|
||||
},
|
||||
{
|
||||
"name": "folding",
|
||||
"version": "1.1.1"
|
||||
},
|
||||
{
|
||||
"name": "argpase",
|
||||
"version": "3.2"
|
||||
},
|
||||
{
|
||||
"name": "catch2",
|
||||
"version": "3.8.1"
|
||||
},
|
||||
{
|
||||
"name": "nlohmann-json",
|
||||
"version": "3.11.3"
|
||||
}
|
||||
]
|
||||
}
|
Reference in New Issue
Block a user