|
Download WHAT_TO_UPLOAD.md from pardhas133/Predictive_maintaimce: direct link, hf CLI and curl.
- Browser
- Download file 8.12 kB
-
https://huggingface.co/spaces/pardhas133/Predictive_maintaimce/resolve/main/WHAT_TO_UPLOAD.md
- Command line
-
hf download hf://spaces/pardhas133/Predictive_maintaimce/WHAT_TO_UPLOAD.md
-
curl -L -o WHAT_TO_UPLOAD.md https://huggingface.co/spaces/pardhas133/Predictive_maintaimce/resolve/main/WHAT_TO_UPLOAD.md
8.12 kB
What to Upload: GitHub vs Hugging Face
Quick Summary
| Destination | What to Upload | How |
|---|---|---|
| GitHub | Everything (entire project folder) | git push |
| HF Dataset | Raw data + train/test splits | Scripts auto-upload |
| HF Model | Trained model file | Scripts auto-upload |
| HF Space | App files (code, Dockerfile, requirements) | Scripts auto-upload |
π¦ GitHub Repository
Upload: Everything in the mlops/ folder
What to include:
mlops/
βββ data/
β βββ engine_data.csv β
Upload
β βββ processed/
β βββ train.csv β
Upload
β βββ test.csv β
Upload
βββ src/
β βββ app.py β
Upload
β βββ config.py β
Upload
β βββ data_prep.py β
Upload
β βββ data_register.py β
Upload
β βββ train.py β
Upload
β βββ inference.py β
Upload
β βββ deploy_to_hf.py β
Upload
β βββ eda.py β
Upload
β βββ hf_data_utils.py β
Upload
β βββ hf_model_utils.py β
Upload
βββ notebooks/ β
Upload (if you have EDA notebooks)
βββ models/
β βββ best_model.joblib β οΈ Optional (large file)
βββ .github/
β βββ workflows/
β βββ pipeline.yml β
Upload (IMPORTANT!)
βββ requirements.txt β
Upload
βββ Dockerfile β
Upload
βββ README.md β
Upload
βββ *.md files β
Upload (documentation)
β DON'T upload:
βββ .venv/ β Skip (virtual environment)
βββ __pycache__/ β Skip (Python cache)
βββ mlruns/ β Skip (MLflow tracking - can be large)
βββ .git/ β Skip (git metadata)
How to Upload to GitHub:
cd /Users/ananttripathi/Desktop/mlops
# Initialize git (if not already done)
git init
# Create .gitignore to exclude large/unnecessary files
cat > .gitignore << EOF
.venv/
__pycache__/
*.pyc
*.pyo
*.pyd
.Python
mlruns/
*.log
.DS_Store
EOF
# Add all files
git add .
# Commit
git commit -m "Initial commit: Predictive Maintenance MLOps Pipeline"
# Add remote (replace with your repo URL)
git remote add origin https://github.com/ananttripathi/engine-predictive-maintenance.git
# Push
git push -u origin main
π€ Hugging Face - Dataset Repo
Repo: ananttripathiak/engine-maintenance-dataset
Upload: Data files only
Files uploaded automatically by scripts:
data/engine_data.csvβ Uploaded asdata/engine_data.csvdata/processed/train.csvβ Uploaded asdata/train.csvdata/processed/test.csvβ Uploaded asdata/test.csv
How to Upload:
Option 1: Run the scripts (automatic)
# Step 1: Register raw data
python src/data_register.py
# Step 2: Prepare and upload train/test
python src/data_prep.py
Option 2: Manual upload via HF Hub
- Go to https://huggingface.co/datasets/ananttripathiak/engine-maintenance-dataset
- Click "Add file" β Upload
data/engine_data.csv - Upload
data/processed/train.csvandtest.csv
π€ Hugging Face - Model Repo
Repo: ananttripathiak/engine-maintenance-model
Upload: Trained model file only
File uploaded automatically:
models/best_model.joblibβ Uploaded asmodel.joblib
How to Upload:
Option 1: Run the training script (automatic)
python src/train.py
# This will:
# 1. Train the model
# 2. Save to models/best_model.joblib
# 3. Upload to HF Model Repo automatically
Option 2: Manual upload via HF Hub
- Go to https://huggingface.co/ananttripathiak/engine-maintenance-model
- Click "Add file" β Upload
models/best_model.joblib - Rename it to
model.joblibin the repo
π€ Hugging Face - Space (Streamlit App)
Repo: ananttripathiak/engine-maintenance-space
Upload: App deployment files
Files uploaded automatically by script:
β
src/app.py (Main Streamlit app)
β
src/inference.py (Inference utilities)
β
src/config.py (Configuration)
β
Dockerfile (Container definition)
β
requirements.txt (Dependencies)
β
README.md (Documentation)
β
Other src/*.py files (If needed by app)
β NOT uploaded (ignored):
βββ data/ β Too large
βββ mlruns/ β MLflow tracking
βββ models/ β Model is in Model Repo
βββ .github/ β GitHub-specific
βββ .venv/ β Virtual environment
How to Upload:
Option 1: Run the deployment script (automatic)
python src/deploy_to_hf.py
# This will:
# 1. Create/update the HF Space
# 2. Upload all deployment files
# 3. Configure it as a Streamlit app
Option 2: Manual upload via HF Hub
- Go to https://huggingface.co/spaces/ananttripathiak/engine-maintenance-space
- Upload files one by one or use HF CLI
π Complete Upload Checklist
β Step 1: GitHub (Manual - Do First)
- Create GitHub repository
- Add
.gitignorefile - Push entire
mlops/folder to GitHub - Add GitHub Secrets (HF_TOKEN, HF_DATASET_REPO, HF_MODEL_REPO, HF_SPACE_REPO)
β Step 2: Hugging Face Dataset (Automatic)
- Set
HF_TOKENenvironment variable - Run
python src/data_register.py(uploads raw data) - Run
python src/data_prep.py(uploads train/test)
β Step 3: Hugging Face Model (Automatic)
- Run
python src/train.py(trains model and uploads to HF)
β Step 4: Hugging Face Space (Automatic)
- Run
python src/deploy_to_hf.py(deploys app to HF Space)
β Step 5: Verify
- Check GitHub repo: https://github.com/ananttripathi/engine-predictive-maintenance
- Check HF Dataset: https://huggingface.co/datasets/ananttripathiak/engine-maintenance-dataset
- Check HF Model: https://huggingface.co/ananttripathiak/engine-maintenance-model
- Check HF Space: https://huggingface.co/spaces/ananttripathiak/engine-maintenance-space
π― Key Differences
| Aspect | GitHub | Hugging Face |
|---|---|---|
| Purpose | Code repository & version control | Data/Model/App hosting |
| What | Entire project (code, data, docs) | Specific artifacts (data/model/app) |
| How | Manual git push |
Automatic via scripts |
| Size | Can be large (includes everything) | Optimized (only what's needed) |
| Access | Public/Private repo | Public datasets/models/spaces |
| CI/CD | GitHub Actions workflow | HF Spaces auto-deploy |
π‘ Pro Tips
- GitHub First: Always push to GitHub first, then run HF scripts
- Use .gitignore: Exclude large files like
mlruns/and.venv/from GitHub - HF Scripts are Smart: They automatically create repos if they don't exist
- Check File Sizes: HF has file size limits, so scripts exclude large files
- GitHub Secrets: Store HF credentials in GitHub Secrets for CI/CD
π¨ Common Mistakes to Avoid
β Don't upload to GitHub:
- Virtual environment (
.venv/) - Large MLflow runs (
mlruns/) - Python cache files (
__pycache__/)
β Don't upload to HF Space:
- Raw data files (too large)
- Model files (use Model Repo instead)
- MLflow tracking data
β Do upload to GitHub:
- All source code
- Configuration files
- Documentation
- GitHub Actions workflow
β Do upload to HF:
- Only what each repo type needs (data β Dataset, model β Model, app β Space)
π Need Help?
- GitHub Issues: Check your repo settings and secrets
- HF Upload Errors: Verify
HF_TOKENis set correctly - File Size Issues: Check HF file size limits (usually 10GB for datasets)