File size: 8,116 Bytes
fc00054
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
# What to Upload: GitHub vs Hugging Face

## Quick Summary

| Destination | What to Upload | How |
|------------|----------------|-----|
| **GitHub** | **Everything** (entire project folder) | `git push` |
| **HF Dataset** | Raw data + train/test splits | Scripts auto-upload |
| **HF Model** | Trained model file | Scripts auto-upload |
| **HF Space** | App files (code, Dockerfile, requirements) | Scripts auto-upload |

---

## πŸ“¦ GitHub Repository

### Upload: **Everything in the `mlops/` folder**

**What to include:**
```
mlops/
β”œβ”€β”€ data/
β”‚   β”œβ”€β”€ engine_data.csv          βœ… Upload
β”‚   └── processed/
β”‚       β”œβ”€β”€ train.csv            βœ… Upload
β”‚       └── test.csv             βœ… Upload
β”œβ”€β”€ src/
β”‚   β”œβ”€β”€ app.py                   βœ… Upload
β”‚   β”œβ”€β”€ config.py                βœ… Upload
β”‚   β”œβ”€β”€ data_prep.py             βœ… Upload
β”‚   β”œβ”€β”€ data_register.py         βœ… Upload
β”‚   β”œβ”€β”€ train.py                 βœ… Upload
β”‚   β”œβ”€β”€ inference.py             βœ… Upload
β”‚   β”œβ”€β”€ deploy_to_hf.py         βœ… Upload
β”‚   β”œβ”€β”€ eda.py                  βœ… Upload
β”‚   β”œβ”€β”€ hf_data_utils.py        βœ… Upload
β”‚   └── hf_model_utils.py       βœ… Upload
β”œβ”€β”€ notebooks/                   βœ… Upload (if you have EDA notebooks)
β”œβ”€β”€ models/
β”‚   └── best_model.joblib        ⚠️ Optional (large file)
β”œβ”€β”€ .github/
β”‚   └── workflows/
β”‚       └── pipeline.yml         βœ… Upload (IMPORTANT!)
β”œβ”€β”€ requirements.txt             βœ… Upload
β”œβ”€β”€ Dockerfile                   βœ… Upload
β”œβ”€β”€ README.md                    βœ… Upload
└── *.md files                   βœ… Upload (documentation)

❌ DON'T upload:
β”œβ”€β”€ .venv/                       ❌ Skip (virtual environment)
β”œβ”€β”€ __pycache__/                 ❌ Skip (Python cache)
β”œβ”€β”€ mlruns/                      ❌ Skip (MLflow tracking - can be large)
└── .git/                        ❌ Skip (git metadata)
```

### How to Upload to GitHub:

```bash
cd /Users/ananttripathi/Desktop/mlops

# Initialize git (if not already done)
git init

# Create .gitignore to exclude large/unnecessary files
cat > .gitignore << EOF
.venv/
__pycache__/
*.pyc
*.pyo
*.pyd
.Python
mlruns/
*.log
.DS_Store
EOF

# Add all files
git add .

# Commit
git commit -m "Initial commit: Predictive Maintenance MLOps Pipeline"

# Add remote (replace with your repo URL)
git remote add origin https://github.com/ananttripathi/engine-predictive-maintenance.git

# Push
git push -u origin main
```

---

## πŸ€— Hugging Face - Dataset Repo

**Repo**: `ananttripathiak/engine-maintenance-dataset`

### Upload: **Data files only**

**Files uploaded automatically by scripts:**
- `data/engine_data.csv` β†’ Uploaded as `data/engine_data.csv`
- `data/processed/train.csv` β†’ Uploaded as `data/train.csv`
- `data/processed/test.csv` β†’ Uploaded as `data/test.csv`

### How to Upload:

**Option 1: Run the scripts (automatic)**
```bash
# Step 1: Register raw data
python src/data_register.py

# Step 2: Prepare and upload train/test
python src/data_prep.py
```

**Option 2: Manual upload via HF Hub**
- Go to https://huggingface.co/datasets/ananttripathiak/engine-maintenance-dataset
- Click "Add file" β†’ Upload `data/engine_data.csv`
- Upload `data/processed/train.csv` and `test.csv`

---

## πŸ€— Hugging Face - Model Repo

**Repo**: `ananttripathiak/engine-maintenance-model`

### Upload: **Trained model file only**

**File uploaded automatically:**
- `models/best_model.joblib` β†’ Uploaded as `model.joblib`

### How to Upload:

**Option 1: Run the training script (automatic)**
```bash
python src/train.py
# This will:
# 1. Train the model
# 2. Save to models/best_model.joblib
# 3. Upload to HF Model Repo automatically
```

**Option 2: Manual upload via HF Hub**
- Go to https://huggingface.co/ananttripathiak/engine-maintenance-model
- Click "Add file" β†’ Upload `models/best_model.joblib`
- Rename it to `model.joblib` in the repo

---

## πŸ€— Hugging Face - Space (Streamlit App)

**Repo**: `ananttripathiak/engine-maintenance-space`

### Upload: **App deployment files**

**Files uploaded automatically by script:**
```
βœ… src/app.py                    (Main Streamlit app)
βœ… src/inference.py             (Inference utilities)
βœ… src/config.py                (Configuration)
βœ… Dockerfile                   (Container definition)
βœ… requirements.txt             (Dependencies)
βœ… README.md                    (Documentation)
βœ… Other src/*.py files         (If needed by app)

❌ NOT uploaded (ignored):
β”œβ”€β”€ data/                       ❌ Too large
β”œβ”€β”€ mlruns/                     ❌ MLflow tracking
β”œβ”€β”€ models/                     ❌ Model is in Model Repo
β”œβ”€β”€ .github/                    ❌ GitHub-specific
└── .venv/                      ❌ Virtual environment
```

### How to Upload:

**Option 1: Run the deployment script (automatic)**
```bash
python src/deploy_to_hf.py
# This will:
# 1. Create/update the HF Space
# 2. Upload all deployment files
# 3. Configure it as a Streamlit app
```

**Option 2: Manual upload via HF Hub**
- Go to https://huggingface.co/spaces/ananttripathiak/engine-maintenance-space
- Upload files one by one or use HF CLI

---

## πŸ“‹ Complete Upload Checklist

### βœ… Step 1: GitHub (Manual - Do First)
- [ ] Create GitHub repository
- [ ] Add `.gitignore` file
- [ ] Push entire `mlops/` folder to GitHub
- [ ] Add GitHub Secrets (HF_TOKEN, HF_DATASET_REPO, HF_MODEL_REPO, HF_SPACE_REPO)

### βœ… Step 2: Hugging Face Dataset (Automatic)
- [ ] Set `HF_TOKEN` environment variable
- [ ] Run `python src/data_register.py` (uploads raw data)
- [ ] Run `python src/data_prep.py` (uploads train/test)

### βœ… Step 3: Hugging Face Model (Automatic)
- [ ] Run `python src/train.py` (trains model and uploads to HF)

### βœ… Step 4: Hugging Face Space (Automatic)
- [ ] Run `python src/deploy_to_hf.py` (deploys app to HF Space)

### βœ… Step 5: Verify
- [ ] Check GitHub repo: https://github.com/ananttripathi/engine-predictive-maintenance
- [ ] Check HF Dataset: https://huggingface.co/datasets/ananttripathiak/engine-maintenance-dataset
- [ ] Check HF Model: https://huggingface.co/ananttripathiak/engine-maintenance-model
- [ ] Check HF Space: https://huggingface.co/spaces/ananttripathiak/engine-maintenance-space

---

## 🎯 Key Differences

| Aspect | GitHub | Hugging Face |
|--------|--------|--------------|
| **Purpose** | Code repository & version control | Data/Model/App hosting |
| **What** | Entire project (code, data, docs) | Specific artifacts (data/model/app) |
| **How** | Manual `git push` | Automatic via scripts |
| **Size** | Can be large (includes everything) | Optimized (only what's needed) |
| **Access** | Public/Private repo | Public datasets/models/spaces |
| **CI/CD** | GitHub Actions workflow | HF Spaces auto-deploy |

---

## πŸ’‘ Pro Tips

1. **GitHub First**: Always push to GitHub first, then run HF scripts
2. **Use .gitignore**: Exclude large files like `mlruns/` and `.venv/` from GitHub
3. **HF Scripts are Smart**: They automatically create repos if they don't exist
4. **Check File Sizes**: HF has file size limits, so scripts exclude large files
5. **GitHub Secrets**: Store HF credentials in GitHub Secrets for CI/CD

---

## 🚨 Common Mistakes to Avoid

❌ **Don't upload to GitHub:**
- Virtual environment (`.venv/`)
- Large MLflow runs (`mlruns/`)
- Python cache files (`__pycache__/`)

❌ **Don't upload to HF Space:**
- Raw data files (too large)
- Model files (use Model Repo instead)
- MLflow tracking data

βœ… **Do upload to GitHub:**
- All source code
- Configuration files
- Documentation
- GitHub Actions workflow

βœ… **Do upload to HF:**
- Only what each repo type needs (data β†’ Dataset, model β†’ Model, app β†’ Space)

---

## πŸ“ž Need Help?

- **GitHub Issues**: Check your repo settings and secrets
- **HF Upload Errors**: Verify `HF_TOKEN` is set correctly
- **File Size Issues**: Check HF file size limits (usually 10GB for datasets)