docs: refine dataset distinction, reference split labeling, checkpoint vs peak metrics, and limitations phrasing
d94c8de verified |
Download README.md from omerfarooq223/FireWatch-AI: direct link, hf CLI and curl.
- Browser
- Download file 9.87 kB
-
https://huggingface.co/omerfarooq223/FireWatch-AI/resolve/main/README.md
- Command line
-
hf download hf://omerfarooq223/FireWatch-AI/README.md
-
curl -L -o README.md https://huggingface.co/omerfarooq223/FireWatch-AI/resolve/main/README.md
9.87 kB
| language: | |
| - en | |
| license: agpl-3.0 | |
| tags: | |
| - vision | |
| - object-detection | |
| - instance-segmentation | |
| - yolo | |
| - yolov8 | |
| - fire-detection | |
| - smoke-detection | |
| - safety | |
| pipeline_tag: object-detection | |
| datasets: | |
| - openrsgis/FASDD | |
| model-index: | |
| - name: FireWatch-AI | |
| results: | |
| - task: | |
| type: object-detection | |
| name: Object Detection | |
| dataset: | |
| name: FASDD_CV (Flame and Smoke Detection Dataset) | |
| type: openrsgis/FASDD | |
| split: validation | |
| metrics: | |
| - name: Box mAP@50 (detector-best.pt, Saved Checkpoint) | |
| type: mAP | |
| value: 79.91 | |
| - name: Box mAP@50-95 (detector-best.pt, Saved Checkpoint) | |
| type: mAP | |
| value: 54.23 | |
| - name: Box Precision (detector-best.pt, Saved Checkpoint) | |
| type: precision | |
| value: 78.84 | |
| - name: Box Recall (detector-best.pt, Saved Checkpoint) | |
| type: recall | |
| value: 70.41 | |
| - task: | |
| type: instance-segmentation | |
| name: Instance Segmentation | |
| dataset: | |
| name: Clean Fire Segmentation (Derived Subset) | |
| type: custom | |
| split: validation | |
| metrics: | |
| - name: Mask mAP@50 (seg-best.pt, Saved Checkpoint) | |
| type: mAP | |
| value: 92.29 | |
| - name: Mask mAP@50-95 (seg-best.pt, Saved Checkpoint) | |
| type: mAP | |
| value: 62.29 | |
| - name: Mask Precision (seg-best.pt, Saved Checkpoint) | |
| type: precision | |
| value: 90.50 | |
| - name: Mask Recall (seg-best.pt, Saved Checkpoint) | |
| type: recall | |
| value: 86.26 | |
| # FireWatch AI — Dual Detection & Segmentation YOLO Models | |
| [](https://huggingface.co/spaces/omerfarooq223/firewatch-yolo-inference) | |
| [](https://github.com/omerfarooq223/FireWatch-AI) | |
| [](https://github.com/ultralytics/ultralytics/blob/main/LICENSE) | |
| This repository provides fine-tuned YOLOv8 weights powering the **FireWatch AI** safety monitoring and incident management framework. | |
| Live inference microservice: [omerfarooq223/firewatch-yolo-inference](https://huggingface.co/spaces/omerfarooq223/firewatch-yolo-inference) | |
| --- | |
| ## Model Checkpoints | |
| This repository hosts two distinct weights files: | |
| 1. **`detector-best.pt`** (~262 MB) | |
| * **Task:** Bounding-box Object Detection | |
| * **Base Architecture:** **YOLOv8l** (`depth=1.0, width=1.0`, nc=2) | |
| * **Classes:** | |
| * `0: fire` | |
| * `1: smoke` | |
| * **Role:** Localizing flame and smoke bounding boxes in surveillance video frames. | |
| 2. **`seg-best.pt`** (~6.7 MB) | |
| * **Task:** Polygon Instance Segmentation | |
| * **Base Architecture:** **YOLOv8n-seg** (`depth=0.33, width=0.25`, nc=1) | |
| * **Classes:** | |
| * `0: fire` | |
| * **Role:** Extracting pixel-level flame boundaries and calculating fire surface area. (Does not segment smoke). | |
| --- | |
| ## Evaluation Metrics (Validation Split) | |
| Below are the exact metrics recorded in the training checkpoint logs. Metrics associated with the saved `best.pt` checkpoint are distinguished from individual peak-epoch measurements: | |
| ### 1. Detection Model (`detector-best.pt` — YOLOv8l) | |
| * **Training duration:** 17 epochs completed (target was 50; training stopped at Epoch 17). | |
| * **Saved Checkpoint (`best.pt` / Epoch 17):** | |
| * **`metrics/mAP50(B)`**: **79.91%** (`0.79907`) | |
| * **`metrics/mAP50-95(B)`**: **54.23%** (`0.54227`, peak fitness for the run) | |
| * **`metrics/precision(B)`**: **78.84%** (`0.78842`) | |
| * **`metrics/recall(B)`**: **70.41%** (`0.70407`) | |
| * `val/box_loss`: `1.08875` | |
| * `val/cls_loss`: `0.87772` | |
| * `val/dfl_loss`: `1.28927` | |
| * **Peak Metric Comparison:** | |
| * Box mAP@50 reached its individual peak at **Epoch 16** with **79.92%** (`0.79921`, with Precision: `78.88%`, Recall: `70.41%`, mAP50-95: `54.15%`), before `best.pt` was finalized at Epoch 17 on overall fitness. | |
| ### 2. Segmentation Model (`seg-best.pt` — YOLOv8n-seg) | |
| * **Training duration:** 20 epochs completed. The saved `best.pt` corresponds to the final epoch evaluated (Epoch 20, fitness `1.40684`). | |
| * **Saved Checkpoint Metrics (Epoch 20):** | |
| * **Mask Metrics (Contour Segmentation):** | |
| * **`metrics/mAP50(M)`**: **92.29%** (`0.92287`) | |
| * **`metrics/mAP50-95(M)`**: **62.29%** (`0.62285`) | |
| * **`metrics/precision(M)`**: **90.50%** (`0.90495`) | |
| * **`metrics/recall(M)`**: **86.26%** (`0.86260`) | |
| * **Bounding Box Metrics:** | |
| * **`metrics/mAP50(B)`**: **96.50%** (`0.96503`) | |
| * **`metrics/mAP50-95(B)`**: **78.40%** (`0.78399`) | |
| * **`metrics/precision(B)`**: **92.11%** (`0.92114`) | |
| * **`metrics/recall(B)`**: **89.65%** (`0.89654`) | |
| * `val/box_loss`: `0.66318` | `val/seg_loss`: `1.44122` | `val/cls_loss`: `0.50684` | `val/dfl_loss`: `0.92107` | |
| --- | |
| ## Dataset Details | |
| The detector and segmentation models were trained on separate dataset configurations: | |
| ### A. Detection Dataset (`detector-best.pt`) | |
| * **Upstream Benchmark:** Trained on the **FASDD (Flame And Smoke Detection Dataset)** — `FASDD_CV` computer vision benchmark subset (*Wang et al., 2023*). | |
| * **Benchmark Size:** **95,314 total images** (full official FASDD_CV benchmark across all splits combined). | |
| * **Reference Upstream Benchmark Splits:** | |
| > *Note: The figures below represent the official published FASDD_CV benchmark partition ratios provided for reference. The exact split counts ingested by the specific training notebook script were managed via `data.yaml` and are not recorded in the checkpoint dictionary.* | |
| * Reference Training Partition (~50%): ~47,657 images | |
| * Reference Validation Partition (~33.3%): ~31,771 images | |
| * Reference Test Partition (~16.7%): ~15,886 images | |
| * **Annotations:** Dual-class bounding box labels for `fire` and `smoke`. | |
| ### B. Segmentation Dataset (`seg-best.pt`) | |
| * **Subset Configuration:** Trained using a derived fire segmentation configuration referenced in the training metadata as `/kaggle/working/clean_fire_seg.yaml`. | |
| * **Subset Size & Splits:** **Not recorded.** The derived dataset files, image manifests, and exact train/val sample counts were generated in an ephemeral Kaggle working environment and are not embedded in the model checkpoint. | |
| * **Target Scope:** Single-class polygon annotations for `fire` only (smoke masks are not included). | |
| --- | |
| ## Training Configuration | |
| | Parameter | Detection (`detector-best.pt`) | Segmentation (`seg-best.pt`) | | |
| | :--- | :--- | :--- | | |
| | **Model Architecture** | YOLOv8l (Large) | YOLOv8n-seg (Nano Segmentation) | | |
| | **Pretrained Checkpoint** | Resumed / fine-tuned from `last.pt` | `yolov8n-seg.pt` | | |
| | **Input Image Size** | 640 × 640 | 640 × 640 | | |
| | **Batch Size** | 16 | 16 | | |
| | **Epochs Trained** | 17 epochs | 20 epochs | | |
| | **Optimizer** | Auto (SGD/AdamW, momentum: 0.937, weight_decay: 0.0005) | Auto (SGD/AdamW, momentum: 0.937, weight_decay: 0.0005) | | |
| | **Dropout** | 0.5 | 0.0 | | |
| | **Hardware** | Kaggle GPU (`device: 0`, GPU model: *not recorded*) | Kaggle GPU (`device: 0`, GPU model: *not recorded*) | | |
| | **Ultralytics Version** | `8.4.41` | `8.4.45` | | |
| | **Run Name / Project** | `yolov8l-FASDD` (`Fire-Smoke-FASDD`) | `final_fire_segmentation` | | |
| --- | |
| ## How to Run Inference | |
| ### Installation | |
| ```bash | |
| pip install ultralytics huggingface_hub opencv-python pillow | |
| ``` | |
| ### Python Inference Example | |
| ```python | |
| from huggingface_hub import hf_hub_download | |
| from ultralytics import YOLO | |
| # 1. Download weights from Hugging Face Hub | |
| det_path = hf_hub_download(repo_id="omerfarooq223/FireWatch-AI", filename="detector-best.pt") | |
| seg_path = hf_hub_download(repo_id="omerfarooq223/FireWatch-AI", filename="seg-best.pt") | |
| # 2. Instantiate YOLO models | |
| det_model = YOLO(det_path) | |
| seg_model = YOLO(seg_path) | |
| image_path = "test_frame.jpg" | |
| # 3. Detect fire and smoke bounding boxes | |
| det_results = det_model.predict(source=image_path, conf=0.40, imgsz=640)[0] | |
| for box in det_results.boxes: | |
| cls_id = int(box.cls[0]) | |
| label = det_results.names[cls_id] | |
| conf = float(box.conf[0]) | |
| xyxy = box.xyxy[0].tolist() | |
| print(f"Detected {label} ({conf:.2%}) at {xyxy}") | |
| # 4. Extract polygon masks for flame regions | |
| seg_results = seg_model.predict(source=image_path, conf=0.35, imgsz=640)[0] | |
| if seg_results.masks is not None: | |
| for polygon in seg_results.masks.xy: | |
| print(f"Fire polygon coordinates count: {len(polygon)}") | |
| ``` | |
| --- | |
| ## Potential Limitations & Operational Considerations | |
| The following items represent theoretical considerations and known architectural constraints; untested scenarios have not been quantitatively evaluated: | |
| * **Potential Optical Confounds:** In vision-based monitoring systems, strong directional solar glare, low-angle sunrise/sunset backlighting, reflections off polished surfaces, or high-intensity industrial sodium lamps may pose risks of false positive activations. These environmental conditions have not been systematically tested. | |
| * **Atmospheric Particulates vs. Smoke:** Thin or diffuse smoke plumes share visual features with dense fog, steam plumes, and suspended dust. Performance in heavy fog or steam environments has not been benchmarked. | |
| * **Low-Light / Nighttime Deployment:** Detection capability in poorly lit conditions without active illumination or complementary thermal/infrared sensors remains unverified. | |
| * **Architectural Scope Constraint:** The segmentation checkpoint (`seg-best.pt`) was trained strictly for flame instance segmentation (`fire`); it does not segment smoke regions. Use `detector-best.pt` when smoke monitoring is needed. | |
| --- | |
| ## Licensing | |
| * **Model Weights:** Ultralytics YOLOv8 weights are distributed under the [AGPL-3.0 License](https://github.com/ultralytics/ultralytics/blob/main/LICENSE). | |
| * **Application Framework:** [MIT License](https://github.com/omerfarooq223/FireWatch-AI/blob/main/LICENSE). | |
| * **FASDD Dataset:** Distributed under academic research and non-commercial benchmarking terms. | |