|
Download WHAT_TO_UPLOAD.md from pardhas133/Predictive_maintaimce: direct link, hf CLI and curl.
- Browser
- Download file 8.12 kB
-
https://huggingface.co/spaces/pardhas133/Predictive_maintaimce/resolve/main/WHAT_TO_UPLOAD.md
- Command line
-
hf download hf://spaces/pardhas133/Predictive_maintaimce/WHAT_TO_UPLOAD.md
-
curl -L -o WHAT_TO_UPLOAD.md https://huggingface.co/spaces/pardhas133/Predictive_maintaimce/resolve/main/WHAT_TO_UPLOAD.md
8.12 kB
| # What to Upload: GitHub vs Hugging Face | |
| ## Quick Summary | |
| | Destination | What to Upload | How | | |
| |------------|----------------|-----| | |
| | **GitHub** | **Everything** (entire project folder) | `git push` | | |
| | **HF Dataset** | Raw data + train/test splits | Scripts auto-upload | | |
| | **HF Model** | Trained model file | Scripts auto-upload | | |
| | **HF Space** | App files (code, Dockerfile, requirements) | Scripts auto-upload | | |
| --- | |
| ## π¦ GitHub Repository | |
| ### Upload: **Everything in the `mlops/` folder** | |
| **What to include:** | |
| ``` | |
| mlops/ | |
| βββ data/ | |
| β βββ engine_data.csv β Upload | |
| β βββ processed/ | |
| β βββ train.csv β Upload | |
| β βββ test.csv β Upload | |
| βββ src/ | |
| β βββ app.py β Upload | |
| β βββ config.py β Upload | |
| β βββ data_prep.py β Upload | |
| β βββ data_register.py β Upload | |
| β βββ train.py β Upload | |
| β βββ inference.py β Upload | |
| β βββ deploy_to_hf.py β Upload | |
| β βββ eda.py β Upload | |
| β βββ hf_data_utils.py β Upload | |
| β βββ hf_model_utils.py β Upload | |
| βββ notebooks/ β Upload (if you have EDA notebooks) | |
| βββ models/ | |
| β βββ best_model.joblib β οΈ Optional (large file) | |
| βββ .github/ | |
| β βββ workflows/ | |
| β βββ pipeline.yml β Upload (IMPORTANT!) | |
| βββ requirements.txt β Upload | |
| βββ Dockerfile β Upload | |
| βββ README.md β Upload | |
| βββ *.md files β Upload (documentation) | |
| β DON'T upload: | |
| βββ .venv/ β Skip (virtual environment) | |
| βββ __pycache__/ β Skip (Python cache) | |
| βββ mlruns/ β Skip (MLflow tracking - can be large) | |
| βββ .git/ β Skip (git metadata) | |
| ``` | |
| ### How to Upload to GitHub: | |
| ```bash | |
| cd /Users/ananttripathi/Desktop/mlops | |
| # Initialize git (if not already done) | |
| git init | |
| # Create .gitignore to exclude large/unnecessary files | |
| cat > .gitignore << EOF | |
| .venv/ | |
| __pycache__/ | |
| *.pyc | |
| *.pyo | |
| *.pyd | |
| .Python | |
| mlruns/ | |
| *.log | |
| .DS_Store | |
| EOF | |
| # Add all files | |
| git add . | |
| # Commit | |
| git commit -m "Initial commit: Predictive Maintenance MLOps Pipeline" | |
| # Add remote (replace with your repo URL) | |
| git remote add origin https://github.com/ananttripathi/engine-predictive-maintenance.git | |
| # Push | |
| git push -u origin main | |
| ``` | |
| --- | |
| ## π€ Hugging Face - Dataset Repo | |
| **Repo**: `ananttripathiak/engine-maintenance-dataset` | |
| ### Upload: **Data files only** | |
| **Files uploaded automatically by scripts:** | |
| - `data/engine_data.csv` β Uploaded as `data/engine_data.csv` | |
| - `data/processed/train.csv` β Uploaded as `data/train.csv` | |
| - `data/processed/test.csv` β Uploaded as `data/test.csv` | |
| ### How to Upload: | |
| **Option 1: Run the scripts (automatic)** | |
| ```bash | |
| # Step 1: Register raw data | |
| python src/data_register.py | |
| # Step 2: Prepare and upload train/test | |
| python src/data_prep.py | |
| ``` | |
| **Option 2: Manual upload via HF Hub** | |
| - Go to https://huggingface.co/datasets/ananttripathiak/engine-maintenance-dataset | |
| - Click "Add file" β Upload `data/engine_data.csv` | |
| - Upload `data/processed/train.csv` and `test.csv` | |
| --- | |
| ## π€ Hugging Face - Model Repo | |
| **Repo**: `ananttripathiak/engine-maintenance-model` | |
| ### Upload: **Trained model file only** | |
| **File uploaded automatically:** | |
| - `models/best_model.joblib` β Uploaded as `model.joblib` | |
| ### How to Upload: | |
| **Option 1: Run the training script (automatic)** | |
| ```bash | |
| python src/train.py | |
| # This will: | |
| # 1. Train the model | |
| # 2. Save to models/best_model.joblib | |
| # 3. Upload to HF Model Repo automatically | |
| ``` | |
| **Option 2: Manual upload via HF Hub** | |
| - Go to https://huggingface.co/ananttripathiak/engine-maintenance-model | |
| - Click "Add file" β Upload `models/best_model.joblib` | |
| - Rename it to `model.joblib` in the repo | |
| --- | |
| ## π€ Hugging Face - Space (Streamlit App) | |
| **Repo**: `ananttripathiak/engine-maintenance-space` | |
| ### Upload: **App deployment files** | |
| **Files uploaded automatically by script:** | |
| ``` | |
| β src/app.py (Main Streamlit app) | |
| β src/inference.py (Inference utilities) | |
| β src/config.py (Configuration) | |
| β Dockerfile (Container definition) | |
| β requirements.txt (Dependencies) | |
| β README.md (Documentation) | |
| β Other src/*.py files (If needed by app) | |
| β NOT uploaded (ignored): | |
| βββ data/ β Too large | |
| βββ mlruns/ β MLflow tracking | |
| βββ models/ β Model is in Model Repo | |
| βββ .github/ β GitHub-specific | |
| βββ .venv/ β Virtual environment | |
| ``` | |
| ### How to Upload: | |
| **Option 1: Run the deployment script (automatic)** | |
| ```bash | |
| python src/deploy_to_hf.py | |
| # This will: | |
| # 1. Create/update the HF Space | |
| # 2. Upload all deployment files | |
| # 3. Configure it as a Streamlit app | |
| ``` | |
| **Option 2: Manual upload via HF Hub** | |
| - Go to https://huggingface.co/spaces/ananttripathiak/engine-maintenance-space | |
| - Upload files one by one or use HF CLI | |
| --- | |
| ## π Complete Upload Checklist | |
| ### β Step 1: GitHub (Manual - Do First) | |
| - [ ] Create GitHub repository | |
| - [ ] Add `.gitignore` file | |
| - [ ] Push entire `mlops/` folder to GitHub | |
| - [ ] Add GitHub Secrets (HF_TOKEN, HF_DATASET_REPO, HF_MODEL_REPO, HF_SPACE_REPO) | |
| ### β Step 2: Hugging Face Dataset (Automatic) | |
| - [ ] Set `HF_TOKEN` environment variable | |
| - [ ] Run `python src/data_register.py` (uploads raw data) | |
| - [ ] Run `python src/data_prep.py` (uploads train/test) | |
| ### β Step 3: Hugging Face Model (Automatic) | |
| - [ ] Run `python src/train.py` (trains model and uploads to HF) | |
| ### β Step 4: Hugging Face Space (Automatic) | |
| - [ ] Run `python src/deploy_to_hf.py` (deploys app to HF Space) | |
| ### β Step 5: Verify | |
| - [ ] Check GitHub repo: https://github.com/ananttripathi/engine-predictive-maintenance | |
| - [ ] Check HF Dataset: https://huggingface.co/datasets/ananttripathiak/engine-maintenance-dataset | |
| - [ ] Check HF Model: https://huggingface.co/ananttripathiak/engine-maintenance-model | |
| - [ ] Check HF Space: https://huggingface.co/spaces/ananttripathiak/engine-maintenance-space | |
| --- | |
| ## π― Key Differences | |
| | Aspect | GitHub | Hugging Face | | |
| |--------|--------|--------------| | |
| | **Purpose** | Code repository & version control | Data/Model/App hosting | | |
| | **What** | Entire project (code, data, docs) | Specific artifacts (data/model/app) | | |
| | **How** | Manual `git push` | Automatic via scripts | | |
| | **Size** | Can be large (includes everything) | Optimized (only what's needed) | | |
| | **Access** | Public/Private repo | Public datasets/models/spaces | | |
| | **CI/CD** | GitHub Actions workflow | HF Spaces auto-deploy | | |
| --- | |
| ## π‘ Pro Tips | |
| 1. **GitHub First**: Always push to GitHub first, then run HF scripts | |
| 2. **Use .gitignore**: Exclude large files like `mlruns/` and `.venv/` from GitHub | |
| 3. **HF Scripts are Smart**: They automatically create repos if they don't exist | |
| 4. **Check File Sizes**: HF has file size limits, so scripts exclude large files | |
| 5. **GitHub Secrets**: Store HF credentials in GitHub Secrets for CI/CD | |
| --- | |
| ## π¨ Common Mistakes to Avoid | |
| β **Don't upload to GitHub:** | |
| - Virtual environment (`.venv/`) | |
| - Large MLflow runs (`mlruns/`) | |
| - Python cache files (`__pycache__/`) | |
| β **Don't upload to HF Space:** | |
| - Raw data files (too large) | |
| - Model files (use Model Repo instead) | |
| - MLflow tracking data | |
| β **Do upload to GitHub:** | |
| - All source code | |
| - Configuration files | |
| - Documentation | |
| - GitHub Actions workflow | |
| β **Do upload to HF:** | |
| - Only what each repo type needs (data β Dataset, model β Model, app β Space) | |
| --- | |
| ## π Need Help? | |
| - **GitHub Issues**: Check your repo settings and secrets | |
| - **HF Upload Errors**: Verify `HF_TOKEN` is set correctly | |
| - **File Size Issues**: Check HF file size limits (usually 10GB for datasets) | |