π€ AutoMLOps
[](https://www.python.org/)
[](https://flask.palletsprojects.com/)
[](https://airflow.apache.org/)
[](https://mlflow.org/)
[](https://www.docker.com/)
[](#)
[](https://huggingface.co/spaces/mnoorchenar/AutoMLOps)
[](https://github.com/mnoorchenar/AutoMLOps)
**π€ AutoMLOps** β A full-stack ML experiment tracking and pipeline orchestration platform. Train 50+ algorithms, run visual DAG pipelines powered by Apache Airflow, and manage models β all in one Docker container deployed to HuggingFace Spaces.
---
## Table of Contents
- [Features](#-features)
- [Architecture](#οΈ-architecture)
- [Getting Started](#-getting-started)
- [Docker Deployment](#-docker-deployment)
- [Pages](#-pages)
- [Pipelines](#-pipelines)
- [ML Algorithms](#-ml-algorithms)
- [Project Structure](#-project-structure)
- [Author](#-author)
- [Contributing](#-contributing)
- [Disclaimer](#οΈ-disclaimer)
- [License](#-license)
---
## β¨ Features
| π¨ Pipeline Studio |
Interactive full-screen DAG canvas with clickable nodes, slide-in config panel, and live execution terminal |
| βοΈ Real Apache Airflow |
Pipelines execute as genuine Airflow DAGs with XCom, TaskInstance tracking, and DagRun polling |
| π€ AutoML Engine |
Automated hyperparameter search across all algorithm categories for classification and regression tasks |
| π MLflow Tracking |
Every training run logs parameters, metrics, and model artifacts to a persistent SQLite-backed MLflow store |
| π¦ Model Registry |
Register, version, and transition models through Staging β Production β Archived lifecycle stages |
| π Theme Toggle |
Dark and light mode with instant CSS variable switching and localStorage persistence |
| π³ Single-Container Deployment |
Flask + Airflow Scheduler + SQLite in one Docker image β no external services required |
---
## ποΈ Architecture
```
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β AutoMLOps β
β β
β βββββββββββββββ ββββββββββββββββββββ βββββββββββββββββββββ β
β β Datasets βββββΆβ MLOps Engine βββββΆβ Flask API β β
β β (sklearn + β β (sklearn / β β Backend β β
β β custom) β β XGBoost / β ββββββββββ¬βββββββββββ β
β βββββββββββββββ β LightGBM / β β β
β β MLP) β ββββββββββΌβββββββββββ β
β ββββββββββββββββββββ β Pipeline Studio β β
β β AutoML Page β β
β βββββββββββββββ ββββββββββββββββββββ β Model Registry β β
β β MLflow DB ββββββ Airflow β βββββββββββββββββββββ β
β β (SQLite) β β Scheduler β β
β βββββββββββββββ β (DAGs / XCom / β β
β β TaskInstance) β β
β ββββββββββββββββββββ β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
```
---
## π Getting Started
### Prerequisites
- Python 3.11+
- Docker (for containerised deployment)
- Git
### Local Installation
```bash
# 1. Clone the repository
git clone https://github.com/mnoorchenar/AutoMLOps.git
cd AutoMLOps
# 2. Create a virtual environment
python -m venv venv
source venv/bin/activate # Windows: venv\Scripts\activate
# 3. Install app dependencies
pip install -r requirements.txt
# 4. Install Apache Airflow with official constraints
AIRFLOW_VERSION=2.10.4
pip install "apache-airflow==${AIRFLOW_VERSION}" \
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-3.11.txt"
# 5. Initialise Airflow metadata DB
export AIRFLOW_HOME=$(pwd)/airflow_home
export AIRFLOW__CORE__DAGS_FOLDER=$(pwd)/dags
airflow db migrate
# 6. Start the Airflow scheduler (background)
airflow scheduler &
# 7. Run the Flask application
python app.py
```
Open your browser at `http://localhost:7860` π
---
## π³ Docker Deployment
```bash
# Build and run
docker build -t automlops .
docker run -p 7860:7860 automlops
# Or deploy directly to HuggingFace Spaces
# Push to a Space with sdk: docker in README frontmatter
```
The Dockerfile builds a single image containing:
- Flask application (served via Gunicorn on port 7860)
- Apache Airflow Scheduler (started by `start.sh`)
- All Python dependencies with Airflow constraint-file pinning
- Pre-initialised Airflow SQLite metadata DB (`airflow db migrate`)
---
## π Pages
| Page | Route | Description | Status |
|------|-------|-------------|--------|
| π¨ Pipeline Studio | `/` | Interactive DAG canvas β click nodes to configure and execute pipelines | β
Live |
| π€ AutoML | `/automl` | Automated algorithm search across 50+ models for any dataset | β
Live |
| π¦ Model Registry | `/models` | Browse registered models, versions, and lifecycle stages | β
Live |
---
## π Pipelines
Three production-quality pipelines are pre-built and immediately executable from the Pipeline Studio:
**Training Pipeline** (`training_pipeline`)
```
Load Data β Validate β Preprocess β Feature Engineering β Train Model β Evaluate β Report β Register β Deploy to Staging
```
**Retraining Pipeline** (`retraining_pipeline`)
```
Drift Detection β Fetch New Data β Merge Datasets β Retrain Champion β A/B Test β Promote to Production
```
**Data Processing Pipeline** (`data_pipeline`)
```
Ingest Raw Data β Clean Data β Encode Features β Scale Features β Save to Feature Store
```
Each pipeline node is clickable in the UI β configurable nodes (dataset picker, algorithm picker) show a purple indicator dot. Execution logs stream live in the built-in terminal panel.
---
## π§ ML Algorithms
```python
# AutoMLOps Algorithm Registry β 50+ algorithms across 2 tasks
ALGORITHMS = {
"classification": {
"Linear Models": ["Logistic Regression", "Logistic Regression (L1)", "Ridge Classifier",
"SGD Classifier", "Passive Aggressive", "Linear Discriminant Analysis"],
"Tree-Based": ["Decision Tree", "Random Forest", "Extra Trees",
"Quadratic Discriminant Analysis"],
"Ensemble / Boosting": ["Gradient Boosting", "AdaBoost", "Bagging Classifier",
"XGBoost", "LightGBM"],
"Support Vector Machines":["SVC (RBF Kernel)", "SVC (Polynomial)", "SVC (Linear)", "LinearSVC"],
"Probabilistic": ["Gaussian Naive Bayes", "Bernoulli Naive Bayes", "Complement Naive Bayes"],
"Instance-Based (KNN)": ["KNN (k=3)", "KNN (k=5)", "KNN (k=9)"],
"Neural Networks": ["MLP (Small)", "MLP (Medium)", "MLP (Deep)"],
},
"regression": {
"Linear Models": ["Linear Regression", "Ridge Regression", "Lasso",
"ElasticNet", "Bayesian Ridge", "Huber Regressor"],
"Tree-Based": ["Decision Tree Regressor", "Random Forest Regressor",
"Extra Trees Regressor"],
"Ensemble / Boosting": ["Gradient Boosting Regressor", "AdaBoost Regressor",
"Bagging Regressor", "XGBoost Regressor", "LightGBM Regressor"],
"Support Vector Machines":["SVR (RBF)", "SVR (Linear)"],
"Instance-Based (KNN)": ["KNN Regressor (k=3)", "KNN Regressor (k=5)"],
"Neural Networks": ["MLP Regressor (Small)", "MLP Regressor (Medium)"],
}
}
```
**Built-in Datasets:**
| Dataset | Task | Samples | Features |
|---------|------|---------|---------|
| Iris Flowers | Classification | 150 | 4 |
| Wine Quality | Classification | 178 | 13 |
| Breast Cancer | Classification | 569 | 30 |
| Diabetes Progression | Regression | 442 | 10 |
| California Housing | Regression | 20,640 | 8 |
---
## π Project Structure
```
AutoMLOps/
β
βββ π dags/ # Apache Airflow DAG files (parsed by scheduler)
β βββ __init__.py
β βββ data_pipeline.py
β βββ retraining_pipeline.py
β βββ training_pipeline.py
βββ π mlops/
β βββ __init__.py
β βββ airflow_runner.py
β βββ algorithms.py
β βββ datasets.py
β βββ trainer.py
βββ π pipelines/
β βββ __init__.py
β βββ dag_engine.py
β βββ pipeline_defs.py
βββ π static/ # CSS, JavaScript & assets
β βββ css/
β βββ js/
βββ π templates/ # HTML templates
β βββ automl.html
β βββ base.html
β βββ dashboard.html
β βββ experiments.html
β βββ models.html
β βββ pipeline.html
β βββ β¦ 1 more
βββ π Dockerfile # Single-container image (Flask + Airflow)
βββ π app.py # Flask application entry point + all API routes
βββ π docs-template.html
βββ π docs.html
βββ π readme-template.md
βββ π requirements.txt # Python dependencies
βββ π start.sh # Startup: Airflow scheduler β Gunicorn Flask
```
---
## π¨βπ» Author
[](https://github.com/mnoorchenar/AutoMLOps/stargazers)
[](https://github.com/mnoorchenar/AutoMLOps/fork)
The name "AutoMLOps" is used purely for academic and research purposes. Any similarity to existing product names or trademarks is entirely coincidental. This project has no affiliation with any commercial entity.
