| --- |
| license: apache-2.0 |
| --- |
| # Model Card for MatroidNN |
|
|
| ## Model Details |
|
|
| ### Model Description |
|
|
| **Model type:** Neural Network with Matroid-based Feature Selection (MatroidNN) |
|
|
| **Version:** 1.0 |
|
|
| **Framework:** PyTorch |
|
|
| **Last updated:** February 27, 2025 |
|
|
| ### Overview |
|
|
| MatroidNN is a neural network architecture that incorporates matroid theory for feature selection. It addresses the challenge of feature redundancy by selecting a maximally independent set of features based on matroid theory principles before training the neural network. |
|
|
| ### Model Architecture |
|
|
| - **Feature Selection Component**: MatroidFeatureSelector using correlation-based dependency analysis |
| - **Neural Network**: 3-layer feedforward network with batch normalization and dropout |
| - **Input**: Varies based on the number of features selected by the matroid selector |
| - **Hidden Layers**: Configurable hidden layer sizes (default 64 → 32) |
| - **Output**: Multi-class classification (configurable number of classes) |
| - **Parameters**: ~5K-10K parameters (varies based on input/output dimensions) |
|
|
| ## Uses |
|
|
| ### Direct Use |
|
|
| MatroidNN is designed for classification tasks where feature redundancy is a potential issue. It's particularly useful for: |
|
|
| - High-dimensional datasets with correlated features |
| - Feature selection in biological/medical data |
| - Financial prediction with multicollinear variables |
| - Any classification task where feature independence is desired |
|
|
| ### Out-of-Scope Use |
|
|
| This model is not intended for: |
| - Regression tasks (without modification) |
| - Time series prediction (without temporal adaptations) |
| - Raw image or text classification (without appropriate feature extraction) |
|
|
| ## Training Data |
|
|
| The model was developed and tested using synthetic data with deliberate feature dependencies. For real-world applications, the model should be retrained on domain-specific data. |
|
|
| ### Training Dataset |
|
|
| - **Type**: Synthetic data with controlled dependencies |
| - **Size**: 1000 samples (default), configurable |
| - **Features**: 20 initial features (default), configurable |
| - **Classes**: 3 classes (default), configurable |
| - **Distribution**: Equal class distribution in the synthetic data |
|
|
| ## Performance |
|
|
| ### Metrics |
|
|
| On synthetic test data with 3 classes: |
| - **Accuracy**: 94.0% |
| - **Macro-average F1-score**: 0.93 |
| - **Per-class metrics**: |
| - Class 0: Precision 0.96, Recall 1.00, F1 0.98 |
| - Class 1: Precision 0.86, Recall 0.86, F1 0.86 |
| - Class 2: Precision 0.97, Recall 0.93, F1 0.95 |
|
|
| ### Factors |
|
|
| Performance may vary based on: |
| - Feature correlation structure in the dataset |
| - Number of initial features and their information content |
| - Class distribution balance |
| - Rank threshold parameter in the MatroidFeatureSelector |
|
|
| ## Limitations |
|
|
| - The matroid-based feature selection uses correlation as a proxy for independence, which may not capture all forms of dependency |
| - The current implementation assumes numerical features and may require adaptation for categorical features |
| - Feature selection is performed once before training and does not adapt during training |
| - The rank threshold parameter requires careful tuning based on the dataset |
|
|
| ## Ethical Considerations |
|
|
| - Feature selection might unintentionally exclude features that are important for fairness considerations |
| - The model inherits any biases present in the training data |
| - Results should be interpreted with caution in high-stakes applications, with human oversight |
|
|
| ## Technical Specifications |
|
|
| ### Hardware Requirements |
|
|
| - Training: CUDA-capable GPU recommended for larger datasets |
| - Inference: CPU sufficient for most applications |
|
|
| ### Software Requirements |
|
|
| - Python 3.8+ |
| - PyTorch 1.8+ |
| - NumPy 1.20+ |
| - scikit-learn 0.24+ |
|
|
| ### Training Hyperparameters |
|
|
| - **Batch size**: 32 (default) |
| - **Learning rate**: 0.001 (default) |
| - **Optimizer**: Adam |
| - **Loss function**: Cross-Entropy Loss |
| - **Epochs**: Early stopping based on validation loss (patience=10) |
| - **Feature selection rank threshold**: 0.7 (default, configurable) |
|
|
| ## How to Use |
|
|
| ```python |
| from matroid_nn import MatroidFeatureSelector, MatroidNN |
| |
| # Initialize feature selector |
| selector = MatroidFeatureSelector(rank_threshold=0.7) |
| |
| # Apply feature selection |
| X_train_selected = selector.fit_transform(X_train) |
| X_test_selected = selector.transform(X_test) |
| |
| # Create and train model |
| model = MatroidNN( |
| input_size=X_train_selected.shape[1], |
| hidden_size=64, |
| output_size=num_classes |
| ) |
