Balanced Bagging Classifier

Task	Classification (imbalanced classes)
Method key (analysis_type)	bbc
Button in the tools panel	Bagging
Prediction on new data	Yes — see Prediction
Library	imblearn.ensemble.BalancedBaggingClassifier with sklearn.tree.DecisionTreeClassifier

When to use

One class has many more samples than another (for example, 40 "negative" and 8 "positive" samples). An ordinary classifier then tends to predict the large class for almost everything and still reports a high accuracy. Balanced bagging trains every tree on a balanced subset of the data, so the small class gets the same weight as the large one.

How it works

Bagging (bootstrap aggregating) trains $T$ models on different random subsets of the training rows and combines them by voting. Balanced bagging adds one step: before each model is trained, the subset is under-sampled with RandomUnderSampler so that the classes have equal size (by default, every class is reduced to the size of the smallest one).

1. For $t = 1 \dots T$: draw a bootstrap sample of the rows, under-sample it to balance the classes, train a decision tree on it.
2. Predict by averaging the class probabilities of the $T$ trees.

Each tree sees only a part of the large class, but together the trees cover most of it.

Parameters

Defaults as set by the analysis dialog.

Parameter	UI label	Default	Notes
n_estimators	Number of Estimators	10	Number of trees $T$
max_samples	Max Samples	1.0	Share of rows drawn for each tree
sampling_strategy	Sampling Strategy	auto	Which classes are under-sampled: auto (all but the smallest), majority, not minority, not majority, all
replacement	Replacement	false	Under-sample with replacement
bootstrap	Bootstrap Samples	on	Draw rows with replacement
random_state	Random State	42
base_estimator.max_depth	Max Depth	unlimited	Depth of each tree
base_estimator.min_samples_split	Min Samples Split	2
base_estimator.min_samples_leaf	Min Samples Leaf	1
max_features	—	1.0	Share of features for each tree. Not in the dialog.
bootstrap_features	—	false	Draw features with replacement. Not in the dialog.
feature_selection	Enable Feature Selection	off	See Feature selection.
average_replicates	Average technical replicates	off	See Averaging replicates.

The model always runs with n_jobs=1.

Preprocessing

No scaling: trees do not need it.

Before the model: replicate averaging and feature selection, if they are on. See Order of steps.

Fallbacks. The analysis tries hard not to fail:

• a CV fold whose training part contains only one class is skipped;
• if balanced bagging fails in a fold, or for the final model, a single DecisionTreeClassifier is used instead, and a warning is written to the server log.

The results do not show that a fallback happened. If cv_scores has fewer values than the number of folds you set, some folds were skipped.

Results and metrics

The common classification metrics are described in Metrics. AUC is not computed for this method. For imbalanced data, look at recall, sensitivity and specificity per class rather than at accuracy.

Method-specific fields:

Field	Meaning
feature_importances	Mean decrease in impurity, averaged over the trees (final model)
parameters	All parameters actually used

Visualizations

The visualization dialog offers no plots for this method in the current version.

Source code

run_bbc_analysis in chrometrica/analysis/analysis.py. The imports, the parameters and the base tree, the model in the CV loop and the final model:

analysis.py · run_bbc_analysis() · lines 5521–5766

def run_bbc_analysis(X, y, parameters, cv_method, cv_folds):
    """Run Balanced Bagging Classifier analysis with cross-validation and optional feature selection"""
    from imblearn.ensemble import BalancedBaggingClassifier
    from sklearn.model_selection import StratifiedKFold, GroupKFold
    from sklearn.metrics import classification_report, confusion_matrix, accuracy_score
    from sklearn.tree import DecisionTreeClassifier
    from imblearn.under_sampling import RandomUnderSampler
    # ...
    n_estimators = parameters.get('n_estimators', 10)
    max_samples = parameters.get('max_samples', 1.0)
    max_features = parameters.get('max_features', 1.0)
    bootstrap = parameters.get('bootstrap', True)
    bootstrap_features = parameters.get('bootstrap_features', False)
    random_state = parameters.get('random_state', 42)

    # Sampling strategy for balancing - ensure we have samples from all classes
    sampling_strategy = parameters.get('sampling_strategy', 'auto')
    replacement = parameters.get('replacement', False)  # Whether to sample with replacement
    # ...
    base_estimator_params = parameters.get('base_estimator', {})
    base_estimator = DecisionTreeClassifier(
        max_depth=base_estimator_params.get('max_depth', None),
        min_samples_split=base_estimator_params.get('min_samples_split', 2),
        min_samples_leaf=base_estimator_params.get('min_samples_leaf', 1),
        random_state=random_state
    # ...
    for train_idx, test_idx in cv.split(X_processed, y, groups=groups_data):
        X_train, X_test = X_processed[train_idx], X_processed[test_idx]
        y_train, y_test = y[train_idx], y[test_idx]

        # Check if we have at least 2 classes in this fold
        unique_classes = np.unique(y_train)
        if len(unique_classes) < 2:
            logger.warning(f"Fold has only {len(unique_classes)} class(es). Skipping this fold.")
            continue

        # Train and predict
        try:
            bbc_cv = BalancedBaggingClassifier(
                estimator=base_estimator,
                n_estimators=n_estimators,
                max_samples=max_samples,
                max_features=max_features,
                bootstrap=bootstrap,
                bootstrap_features=bootstrap_features,
                random_state=random_state,
                sampling_strategy=sampling_strategy,
                replacement=replacement,
                n_jobs=1  # Set to 1 to avoid multiprocessing issues
            )
            bbc_cv.fit(X_train, y_train)
            y_pred = bbc_cv.predict(X_test)
    # ...
    # Fit final model on all data
    try:
        bbc = BalancedBaggingClassifier(
            estimator=base_estimator,
            n_estimators=n_estimators,
            max_samples=max_samples,
            max_features=max_features,
            bootstrap=bootstrap,
            bootstrap_features=bootstrap_features,
            random_state=random_state,
            sampling_strategy=sampling_strategy,
            replacement=replacement,
            n_jobs=1
        )
        bbc.fit(X_processed, y)
        y_pred_full = bbc.predict(X_processed)

Full source of run_bbc_analysis()

analysis.py · run_bbc_analysis() · lines 5521–5928

def run_bbc_analysis(X, y, parameters, cv_method, cv_folds):
    """Run Balanced Bagging Classifier analysis with cross-validation and optional feature selection"""
    from imblearn.ensemble import BalancedBaggingClassifier
    from sklearn.model_selection import StratifiedKFold, GroupKFold
    from sklearn.metrics import classification_report, confusion_matrix, accuracy_score
    from sklearn.tree import DecisionTreeClassifier
    from imblearn.under_sampling import RandomUnderSampler
    import numpy as np

    if parameters.get('average_replicates', False):
        groups = parameters.get('groups')
        if groups is None:
            raise ValueError("'groups' must be provided when averaging replicates is enabled.")
        X, y, new_groups = average_replicates(X, y, groups, method='mean')
        # Update parameters with new groups (now each row is unique)
        parameters['groups'] = new_groups
        # If you used group CV, you might want to keep it; but with unique groups,
        # GroupKFold becomes equivalent to standard KFold. You can leave it as is.


    # Get overall classes for consistent confusion matrix dimensions
    overall_classes = np.unique(y)
    overall_classes_list = overall_classes.tolist()
    n_overall_classes = len(overall_classes_list)

    # Check for feature selection
    feature_selection_params = get_feature_selection_params(parameters)
    if feature_selection_params:
        X_processed, selected_features, feature_scores = select_features(
            X, y, feature_selection_params, 'classification'
        )
    else:
        X_processed = X
        selected_features = list(range(X.shape[1]))
        feature_scores = None

    # Get BBC parameters from analysis parameters
    n_estimators = parameters.get('n_estimators', 10)
    max_samples = parameters.get('max_samples', 1.0)
    max_features = parameters.get('max_features', 1.0)
    bootstrap = parameters.get('bootstrap', True)
    bootstrap_features = parameters.get('bootstrap_features', False)
    random_state = parameters.get('random_state', 42)

    # Sampling strategy for balancing - ensure we have samples from all classes
    sampling_strategy = parameters.get('sampling_strategy', 'auto')
    replacement = parameters.get('replacement', False)  # Whether to sample with replacement

    # Use undersampler with replacement to ensure we always have all classes
    # This creates a sampler that will work even with small class samples
    sampler = RandomUnderSampler(
        sampling_strategy=sampling_strategy,
        random_state=random_state,
        replacement=replacement  # Allow replacement when classes are very small
    )

    # Base estimator for BBC (default is DecisionTreeClassifier)
    base_estimator_params = parameters.get('base_estimator', {})
    base_estimator = DecisionTreeClassifier(
        max_depth=base_estimator_params.get('max_depth', None),
        min_samples_split=base_estimator_params.get('min_samples_split', 2),
        min_samples_leaf=base_estimator_params.get('min_samples_leaf', 1),
        random_state=random_state
    )

    # Enhanced cross-validation with metrics
    cv_scores = []
    cv_predictions = []  # Store predictions from each fold
    cv_true_labels = []  # Store true labels from each fold
    cv_class_reports = []  # Store classification reports from each fold
    cv_confusion_matrices = []  # Store confusion matrices for each fold (using overall classes)

    # Sensitivity and specificity tracking for binary classification
    cv_sensitivity_scores = []  # Store sensitivity from each fold (for binary)
    cv_specificity_scores = []  # Store specificity from each fold (for binary)

    if cv_method == 'stratified':
        cv = StratifiedKFold(n_splits=cv_folds, shuffle=True, random_state=random_state)
    elif cv_method == 'group':
        groups = parameters.get('groups')
        if groups is None:
            raise ValueError("Groups parameter required for group cross-validation")
        cv = GroupKFold(n_splits=cv_folds)
    else:
        # Simple K-fold
        from sklearn.model_selection import KFold
        cv = KFold(n_splits=cv_folds, shuffle=True, random_state=random_state)

    # For group CV, we need to handle groups in the loop
    groups_data = parameters.get('groups') if cv_method == 'group' else None

    for train_idx, test_idx in cv.split(X_processed, y, groups=groups_data):
        X_train, X_test = X_processed[train_idx], X_processed[test_idx]
        y_train, y_test = y[train_idx], y[test_idx]

        # Check if we have at least 2 classes in this fold
        unique_classes = np.unique(y_train)
        if len(unique_classes) < 2:
            logger.warning(f"Fold has only {len(unique_classes)} class(es). Skipping this fold.")
            continue

        # Train and predict
        try:
            bbc_cv = BalancedBaggingClassifier(
                estimator=base_estimator,
                n_estimators=n_estimators,
                max_samples=max_samples,
                max_features=max_features,
                bootstrap=bootstrap,
                bootstrap_features=bootstrap_features,
                random_state=random_state,
                sampling_strategy=sampling_strategy,
                replacement=replacement,
                n_jobs=1  # Set to 1 to avoid multiprocessing issues
            )
            bbc_cv.fit(X_train, y_train)
            y_pred = bbc_cv.predict(X_test)

            # Store fold results
            cv_scores.append(accuracy_score(y_test, y_pred))
            cv_predictions.extend(y_pred)
            cv_true_labels.extend(y_test)

            # Store per-fold classification report
            fold_report = classification_report(y_test, y_pred, output_dict=True, zero_division=0)
            cv_class_reports.append(fold_report)

            # Create confusion matrix using overall classes to ensure consistent dimensions
            cm = confusion_matrix(y_test, y_pred, labels=overall_classes_list)
            cv_confusion_matrices.append(cm)

            # Calculate sensitivity and specificity for binary classification using the full 2x2 matrix
            if n_overall_classes == 2:
                tn, fp, fn, tp = cm.ravel()
                sensitivity = tp / (tp + fn) if (tp + fn) > 0 else 0
                specificity = tn / (tn + fp) if (tn + fp) > 0 else 0
                cv_sensitivity_scores.append(sensitivity)
                cv_specificity_scores.append(specificity)
            else:
                # For multiclass, we can't calculate single sensitivity/specificity values per fold
                cv_sensitivity_scores.append(None)
                cv_specificity_scores.append(None)

        except Exception as e:
            logger.warning(f"Error in BBC fold: {e}. Skipping this fold.")
            # Fallback to regular DecisionTreeClassifier for this fold
            try:
                dt = DecisionTreeClassifier(random_state=random_state)
                dt.fit(X_train, y_train)
                y_pred = dt.predict(X_test)

                # Store fold results
                cv_scores.append(accuracy_score(y_test, y_pred))
                cv_predictions.extend(y_pred)
                cv_true_labels.extend(y_test)

                fold_report = classification_report(y_test, y_pred, output_dict=True, zero_division=0)
                cv_class_reports.append(fold_report)

                # Create confusion matrix using overall classes
                cm = confusion_matrix(y_test, y_pred, labels=overall_classes_list)
                cv_confusion_matrices.append(cm)

                # Calculate sensitivity and specificity
                if n_overall_classes == 2:
                    tn, fp, fn, tp = cm.ravel()
                    sensitivity = tp / (tp + fn) if (tp + fn) > 0 else 0
                    specificity = tn / (tn + fp) if (tn + fp) > 0 else 0
                    cv_sensitivity_scores.append(sensitivity)
                    cv_specificity_scores.append(specificity)
                else:
                    cv_sensitivity_scores.append(None)
                    cv_specificity_scores.append(None)
            except:
                continue

    # If no folds were successful, raise an error
    if len(cv_scores) == 0:
        raise ValueError("BBC analysis failed on all CV folds. Check if your dataset has sufficient samples per class.")

    # Calculate CV-based metrics
    cv_precision_scores = []
    cv_recall_scores = []
    cv_f1_scores = []

    # Track if this is binary classification
    is_binary = n_overall_classes == 2

    for report in cv_class_reports:
        # Extract macro averages from each fold
        if 'macro avg' in report:
            cv_precision_scores.append(report['macro avg']['precision'])
            cv_recall_scores.append(report['macro avg']['recall'])
            cv_f1_scores.append(report['macro avg']['f1-score'])

    # Calculate overall CV confusion matrix (aggregated)
    cv_confusion_matrix_aggregated = confusion_matrix(cv_true_labels, cv_predictions,
                                                      labels=overall_classes_list).tolist()

    # Calculate overall CV sensitivity and specificity from aggregated confusion matrix
    if is_binary:
        cm_agg = np.array(cv_confusion_matrix_aggregated)
        tn, fp, fn, tp = cm_agg.ravel()
        cv_sensitivity_aggregated = tp / (tp + fn) if (tp + fn) > 0 else 0
        cv_specificity_aggregated = tn / (tn + fp) if (tn + fp) > 0 else 0
    else:
        cv_sensitivity_aggregated = None
        cv_specificity_aggregated = None

    # Calculate average confusion matrix using the outer function
    if cv_confusion_matrices:
        cv_confusion_matrix_avg, avg_matrix_classes = create_average_confusion_matrix(
            cv_confusion_matrices, [overall_classes_list] * len(cv_confusion_matrices)
        )

        # Calculate average sensitivity and specificity from averaged confusion matrix (binary only)
        if is_binary:
            cm_avg = np.array(cv_confusion_matrix_avg)
            tn, fp, fn, tp = cm_avg.ravel()
            cv_sensitivity_avg = tp / (tp + fn) if (tp + fn) > 0 else 0
            cv_specificity_avg = tn / (tn + fp) if (tn + fp) > 0 else 0
        else:
            cv_sensitivity_avg = None
            cv_specificity_avg = None
    else:
        cv_confusion_matrix_avg = []
        avg_matrix_classes = []
        cv_sensitivity_avg = None
        cv_specificity_avg = None

    # Fit final model on all data
    try:
        bbc = BalancedBaggingClassifier(
            estimator=base_estimator,
            n_estimators=n_estimators,
            max_samples=max_samples,
            max_features=max_features,
            bootstrap=bootstrap,
            bootstrap_features=bootstrap_features,
            random_state=random_state,
            sampling_strategy=sampling_strategy,
            replacement=replacement,
            n_jobs=1
        )
        bbc.fit(X_processed, y)
        y_pred_full = bbc.predict(X_processed)
    except Exception as e:
        logger.warning(f"Error fitting final BBC model: {e}. Using DecisionTreeClassifier as fallback.")
        # Fallback to regular DecisionTreeClassifier
        bbc = DecisionTreeClassifier(random_state=random_state)
        bbc.fit(X_processed, y)
        y_pred_full = bbc.predict(X_processed)

    # Get feature importances if available
    feature_importances = None
    if hasattr(bbc, 'feature_importances_'):
        feature_importances = bbc.feature_importances_.tolist()
    else:
        # Try to get average feature importance from base estimators
        try:
            if hasattr(bbc, 'estimators_') and bbc.estimators_:
                importances = np.zeros(X_processed.shape[1])
                valid_estimators = 0
                for estimator in bbc.estimators_:
                    if hasattr(estimator, 'feature_importances_'):
                        importances += estimator.feature_importances_
                        valid_estimators += 1
                if valid_estimators > 0:
                    feature_importances = (importances / valid_estimators).tolist()
        except:
            feature_importances = None

    # Get the final model's class order
    final_classes = overall_classes_list

    # Reorder the average confusion matrix to match final_classes if needed
    if cv_confusion_matrix_avg and avg_matrix_classes != final_classes:
        # Create mapping from avg_matrix_classes to final_classes
        avg_to_final = {cls: idx for idx, cls in enumerate(avg_matrix_classes)}

        # Create reordered confusion matrix
        n_classes = len(final_classes)
        reordered_matrix = np.zeros((n_classes, n_classes))

        for i, true_cls in enumerate(final_classes):
            for j, pred_cls in enumerate(final_classes):
                if true_cls in avg_to_final and pred_cls in avg_to_final:
                    orig_i = avg_to_final[true_cls]
                    orig_j = avg_to_final[pred_cls]
                    if (orig_i < len(cv_confusion_matrix_avg) and
                            orig_j < len(cv_confusion_matrix_avg[0])):
                        reordered_matrix[i, j] = cv_confusion_matrix_avg[orig_i][orig_j]

        cv_confusion_matrix_avg = reordered_matrix.tolist()
        avg_matrix_classes = final_classes

    # Calculate sensitivity and specificity for full model (self-test)
    full_confusion_matrix = confusion_matrix(y, y_pred_full, labels=final_classes)
    if is_binary and full_confusion_matrix.shape == (2, 2):
        tn, fp, fn, tp = full_confusion_matrix.ravel()
        full_sensitivity = tp / (tp + fn) if (tp + fn) > 0 else 0
        full_specificity = tn / (tn + fp) if (tn + fp) > 0 else 0
    else:
        full_sensitivity = None
        full_specificity = None

    # Validate consistency between metrics
    validation_message = None
    if is_binary and full_sensitivity is not None and full_specificity is not None:
        avg_sens_spec = (full_sensitivity + full_specificity) / 2
        accuracy = accuracy_score(y, y_pred_full)
        accuracy_diff = abs(accuracy - avg_sens_spec)
        if accuracy_diff > 0.01:  # Allow small floating point differences
            validation_message = f"Note: Accuracy ({accuracy:.3f}) differs from (sensitivity + specificity)/2 ({avg_sens_spec:.3f}) by {accuracy_diff:.3f}"

    # Calculate CV sensitivity mean and std (filtering out None values)
    cv_sensitivity_scores_filtered = [s for s in cv_sensitivity_scores if s is not None]
    if cv_sensitivity_scores_filtered:
        cv_sensitivity_mean = np.mean(cv_sensitivity_scores_filtered)
        cv_sensitivity_std = np.std(cv_sensitivity_scores_filtered)
    else:
        cv_sensitivity_mean = None
        cv_sensitivity_std = None

    # Calculate CV specificity mean and std (filtering out None values)
    cv_specificity_scores_filtered = [s for s in cv_specificity_scores if s is not None]
    if cv_specificity_scores_filtered:
        cv_specificity_mean = np.mean(cv_specificity_scores_filtered)
        cv_specificity_std = np.std(cv_specificity_scores_filtered)
    else:
        cv_specificity_mean = None
        cv_specificity_std = None

    cv_confusion_matrices_lists = [cm.tolist() for cm in cv_confusion_matrices]

    results = {
        'method': 'Balanced Bagging Classifier',
        'cv_scores': cv_scores,
        'cv_mean': np.mean(cv_scores) if cv_scores else 0.0,
        'cv_std': np.std(cv_scores) if cv_scores else 0.0,
        'cv_precision': np.mean(cv_precision_scores) if cv_precision_scores else 0.0,
        'cv_precision_std': np.std(cv_precision_scores) if cv_precision_scores else 0.0,
        'cv_recall': np.mean(cv_recall_scores) if cv_recall_scores else 0.0,
        'cv_recall_std': np.std(cv_recall_scores) if cv_recall_scores else 0.0,
        'cv_f1': np.mean(cv_f1_scores) if cv_f1_scores else 0.0,
        'cv_f1_std': np.std(cv_f1_scores) if cv_f1_scores else 0.0,
        # Sensitivity metrics
        'cv_sensitivity': cv_sensitivity_mean,
        'cv_sensitivity_std': cv_sensitivity_std,
        'cv_sensitivity_aggregated': cv_sensitivity_aggregated,
        'cv_sensitivity_avg': cv_sensitivity_avg,
        'cv_sensitivity_scores': cv_sensitivity_scores,  # Per-fold sensitivities
        # Specificity metrics
        'cv_specificity': cv_specificity_mean,
        'cv_specificity_std': cv_specificity_std,
        'cv_specificity_aggregated': cv_specificity_aggregated,
        'cv_specificity_avg': cv_specificity_avg,
        'cv_specificity_scores': cv_specificity_scores,  # Per-fold specificities
        # Validation flag
        'is_binary': is_binary,
        'validation_message': validation_message,
        # Full model metrics
        'sensitivity': full_sensitivity,
        'specificity': full_specificity,
        # Existing metrics
        'cv_confusion_matrix': cv_confusion_matrix_avg,
        'cv_confusion_matrices': cv_confusion_matrices_lists,
        'cv_confusion_matrix_aggregated': cv_confusion_matrix_aggregated,
        'cv_class_reports': cv_class_reports,
        'classification_report': classification_report(y, y_pred_full, output_dict=True, zero_division=0),
        'confusion_matrix': full_confusion_matrix.tolist(),
        'feature_importances': feature_importances,
        'labels': y.tolist(),
        'predictions': y_pred_full.tolist(),
        'classes': final_classes,
        'parameters': {
            'n_estimators': n_estimators,
            'max_samples': max_samples,
            'max_features': max_features,
            'bootstrap': bootstrap,
            'bootstrap_features': bootstrap_features,
            'sampling_strategy': sampling_strategy,
            'replacement': replacement,
            'base_estimator': str(base_estimator)
        }
    }

    # Add metric consistency check for binary classification
    if is_binary and full_sensitivity is not None and full_specificity is not None:
        results['metric_consistency_check'] = {
            'accuracy': accuracy_score(y, y_pred_full),
            'sensitivity': full_sensitivity,
            'specificity': full_specificity,
            'avg_sens_spec': (full_sensitivity + full_specificity) / 2,
            'difference': abs(accuracy_score(y, y_pred_full) - (full_sensitivity + full_specificity) / 2)
        }

    # Add feature selection info if used
    if feature_selection_params:
        results['feature_selection'] = {
            'selected_features': selected_features,
            'feature_scores': feature_scores,
            'method': feature_selection_params.get('method', 'anova'),
            'k': feature_selection_params.get('k', 'all')
        }

    # Final fitted model for inference on unknown samples (popped in run_analysis)
    return attach_model_bundle(results, bbc, selected_features, feature_selection_params)

Limitations and common pitfalls

• Few trees. The default of 10 trees is small; with strong under-sampling each tree sees only a few rows of the large class. Try 50–100.
• Very small minority class. With 2–3 rows in the smallest class, every tree is trained on 4–6 rows. The model is then very noisy.
• Silent fallbacks. See Preprocessing.

References

• Breiman L. Bagging predictors. Machine Learning, 24, 123–140 (1996). doi:10.1007/BF00058655
• Lemaître G., Nogueira F., Aridas C. K. Imbalanced-learn: a Python toolbox to tackle the curse of imbalanced datasets in machine learning. Journal of Machine Learning Research, 18(17), 1–5 (2017).
• imbalanced-learn user guide: Bagging of balanced samples.