Spaces:
Sleeping
Sleeping
Commit History
-
fix: pass ruff lint+format, clean README, remove stray script
Priya Monishafa0ce5b -
fix: production audit — security hardening + observability + code quality
Priya MonishaC-1 api/routes/classify.py: add missing logger (was NameError on unknown class) M-1 src/models/base_classifier.py: remove print() — logger.info already covers epoch progress m-1 src/data/dataset.py: assert -> ValueError for invalid split param SEC config.py + redis_cache.py + api/main.py: Redis password support (REDIS_PASSWORD env) SEC docker-compose.yml: conditional Redis auth (no-op when REDIS_PASSWORD empty) SEC docker-compose.airflow.yml: replace hardcoded airflow:airflow/admin:admin with env vars OBS api/prometheus_metrics.py: add http_errors Counter + models_loaded Gauge OBS api/main.py: middleware counts 4xx/5xx; models_loaded.set(1) on successful startup DOC .env.example: document REDIS_PASSWORD, AIRFLOW_DB_PASSWORD, AIRFLOW_ADMIN_PASSWORD6319415
-
results: Section 5 complete — all 4 CNNs trained, ResNet50 best at 65.5%
Priya MonishaSection 5 final results: - VGG16: 59.5% (architecture ceiling) - MobileNetV2: 56.7% (backbone lr=1e-5 too conservative) - EfficientNetB0: 58.9% (backbone lr=1e-5 too conservative) - ResNet50: 65.5% test / 63.6% val (data volume bottleneck, 1464 params/img) Pipeline improvements committed alongside results: - dataset.py: create_stratified_split() — random stratified resplit, no sequential bias - loader.py: quality gates 48px→80px, 2% area; compute_overlap_ratio; check_person_overlap - model_factory.py: unfreeze_efficientnet_phase2() added - 01_data_acquisition.py: chair person-overlap filter, MAX_ITER 80k→150k Next: Section 6 YOLOv8 detection training1a28594
-
fix: MobileNetV2 Round 2 overfitting -- Section 5 Round 3 prep
Priya Monisha27.4pp train/val gap. Root cause: 552 params/training-img at features[14:] unfreeze on 3,080 samples. Realistic R3 target: 68-74%. metrics.json: stale fast_mode artifact replaced with confirmed Colab Round 2 values (62.3%, 23ep). Realistic accuracy target documented. augmentor.py: RandomGrayscale(p=0.05) + RandomPerspective(scale=0.2,p=0.3) ColorJitter 0.2->0.3, rotation 15->20deg, ZoomOut (1.3)->(1.4) model_factory.py: get_model() dispatcher + _build_* internal functions MobileNet: full classifier replaced (Dropout 0.2->0.4) unfreeze_mobilenet_phase2: features[14:] slice -> enumerate idx>=16 ~1.23M unfrozen (401/img) vs 1.71M (552/img) Round 2 _build_resnet50: freeze ALL then replace fc (unambiguous ownership) Removed from API: freeze_resnet50_phase1, freeze_vgg16_phase1, unfreeze_vgg16_phase2, count_trainable_params notebook: MobileNet block rewritten AdamW replaces Adam; Phase 1 wd 1e-4->1e-3 Phase 2 backbone lr=1e-5 (was 1e-4) -- highest-impact change Ph…58027ad
-
section-5: fix all RCAs (BN stats, ZoomOut order, fc unfreeze, grad_clip, checkpoint format)
Priya Monisha4e27aff
-
section-5: CNN training framework — base_classifier, model_factory, 04_train_classifier
Priya Monishasrc/models/base_classifier.py: - train_one_epoch(): single epoch, supports GradScaler for mixed precision (EfficientNet) - evaluate(): accuracy, precision, recall, F1 (macro) via sklearn - train(): full loop with early stopping, saves best state_dict (Rule 5) - save_model() / load_model(): state_dict with weights_only=True (Rule 36) - benchmark_inference(): warmup + timed n=100 runs, returns ms/image src/models/model_factory.py: - get_model(name): vgg16, resnet50, mobilenet, efficientnet → 25-class head - VGG16: features frozen, classifier[-1] replaced (4096→25) - ResNet50: fc replaced (2048→25), caller manages 2-phase freeze/unfreeze - MobileNetV2: features frozen, classifier[-1] (1280→25) - EfficientNetB0: features frozen, classifier[-1] (1280→25) - freeze_resnet50_phase1() / unfreeze_resnet50_phase2() helpers - count_trainable_params() notebooks/04_train_classifier.py: - Parameterized: set MODEL = "vgg16"|"resnet50"|"mobilenet"|"efficientnet", run once each - FAST_MODE=False local…fecb3f1 -
fix: RandomZoomOut fill=128 (mid-grey) — black fill normalized to [-2.12,-2.04,-1.80] creates artificial boundary signal; mid-grey normalizes to ~0
Priya Monishafebce91 -
section-4: preprocessing + augmentation — verified end-to-end
Priya Monishasrc/data/preprocessor.py: - validate_yolo_annotations(): 1550 train + 375 val files, 13758 annotations all valid Fixed: assert <= 1.0 not < 1.0 (bbox_to_yolo clamps to min(1.0,...) so boundary values valid) - create_yolo_data_yaml(): absolute paths, works in terminal + Colab - verify_dataset_structure(): counts per split, detects image/label mismatches src/data/augmentor.py: - get_train_transforms(): flip, rotate±15, colorjitter, zoomout, resize 224, ToImage, Normalize(ImageNet) - get_eval_transforms(): resize + normalize only - denormalize(): for visualization - torch wrapped in try/except (local fbgemm.dll issue) notebooks/03_preprocessing.py: - Dataset verified: 2500 classification, 1925 detection, labels match images - YOLO annotations validated: 13758 annotations, all in valid range - data.yaml regenerated with current absolute paths - Augmented sample images saved to docs/figures/ - Tensor shapes confirmed: (3, 224, 224) float32, range ~[-2.1, 2.6] after ImageNet norm2ea8a2d -
fix: remove emoji from loader.py print (Windows cp1252 encoding error)
Priya Monisha261ffe4 -
section-2: dataset acquisition — loader, dataset, helpers, 01_data_acquisition notebook
Priya MonishaKey corrections from mentor notebook analysis: - config.py: replaced COCO original IDs (1-90) with HF 0-indexed IDs (0-79) via SELECTED_CLASSES dict and HF_CATEGORY_TO_CLASS_IDX reverse mapping - Fixed to 25 classes (mentor's notebook had 26 by accidentally including 'train' vehicle) - Confirmed bbox format: [x,y,w,h] absolute pixels (not [xmin,ymin,xmax,ymax]) - No trust_remote_code needed for detection-datasets/coco src/utils/helpers.py: - NumpyEncoder (torch.Tensor → np.bool_ → np.integer → np.floating → np.ndarray) - save_json/load_json with NumpyEncoder - HF Hub helpers: upload_model_to_hub, download_model_from_hub (HfApi().upload_file) - create_hub_repo with exist_ok=True src/data/loader.py: - verify_coco_mapping() using SELECTED_CLASSES + HF_CATEGORY_TO_CLASS_IDX - Checkpoint/resume (mentor had none — Colab disconnect = start over) - save_classification_crop: [x,y,w,h] → PIL.crop((x,y,x+w,y+h)) → resize 224×224 - save_detection_sample: [x,y,w,h] → YOLO normalized format - Pr…9b0ae38 -
section-0+1: project foundation — config.py, requirements, gitignore, project-docs
Priya Monisha- .claude/settings.json: permission allowlist + PostToolUse git-commit hook - .claude/rules/: no_model_in_pages, fast_mode, one_model_per_turn - config.py: Pydantic Settings + all 25 CLASSES with COCO ID comments + 38-rule reference - requirements.txt / requirements-docker.txt / requirements-hf.txt (CPU-only PyTorch index-url) - .gitignore: anchored /data/raw/, /models/*.pt, notebooks/*.ipynb, /runs/ - pyrightconfig.json: extraPaths=["."] for Pylance import resolution - All src/ and api/ __init__.py files - project-docs: 38 rules table, progress tracker, locked decisions - CLAUDE_CONTEXT.md: full session bootstrap contextd091717