mfg-svm-casting-defect-1.5m

A Small Vision Model (SVM) that classifies top-view images of cast submersible-pump impellers as defective or acceptable. It is built on MobileNetV3-Small, has 1.52M parameters, and runs in about 7 ms per image on a single CPU thread.

Part of the Industrial Small Models collection.

Scope: this model has only been trained and tested on one part type (cast pump impellers), photographed in one setup. It is not a general manufacturing-defect detector, and it has not been validated on real factory images.

Licence: non-commercial, demonstration and research use only. The training data is licensed CC BY-NC-ND 4.0, and this model is released under the same licence. See Licence and permitted use.

Headline results

On a leak-free test set of 548 images (366 defective, 182 acceptable), never used for training, tuning or model selection:

Metric Value
Accuracy 99.82%
Defect recall (defective parts caught) 99.73% (365 / 366)
Defect precision 100%
False rejection rate (good parts rejected) 0.00% (0 / 182)
Macro-F1 0.9979
ROC-AUC 1.0000

Compared with standard architectures

All models were trained with the same pipeline and evaluated on the same leak-free test set. Latency is batch size 1, 300 Γ— 300 input, median of 200 runs on an idle Intel Core i7-10700 (CPU) and an NVIDIA TITAN RTX (GPU), in PyTorch.

Model Params Size Missed defects False rejections CPU, 1 thread GPU
This model (MobileNetV3-Small) 1.52M 5.9 MB 1 0 7.0 ms 3.6 ms
ResNet18 (reference) 11.18M 42.7 MB 1 0 69.8 ms 1.8 ms
EfficientNet-B0 4.01M 15.6 MB 0 0 48.4 ms 5.8 ms
ShuffleNetV2 0.5Γ— 0.34M 1.4 MB 6 0 9.7 ms 4.4 ms
  • vs ResNet18: same accuracy with 7.4Γ— fewer parameters, a 7Γ— smaller file and 10Γ— faster CPU inference.
  • vs EfficientNet-B0: within one test image on defect recall, with 6.9Γ— faster CPU inference and 2.6Γ— fewer parameters.
  • vs ShuffleNetV2 0.5Γ—: 5 fewer missed defects, and still faster on CPU.
  • On a large GPU, ResNet18 is faster (1.8 ms vs 3.6 ms). This model is designed for CPU and edge inference, where it is the fastest of the four.

The single-image difference against EfficientNet-B0 is not statistically meaningful. All four models were trained with one seed each.

Files

File Format Size Use
model_fp32.onnx ONNX, FP32 5.8 MB Recommended for deployment. Identical predictions to PyTorch on all 548 test images.
model_fp16.onnx ONNX, FP16 weights, FP32 inputs/outputs 2.9 MB For storage-limited devices. 1 of 548 test predictions differs from FP32 (a borderline case).
best.pt PyTorch state_dict 5.9 MB Reference checkpoint for retraining, fine-tuning and Grad-CAM.

ONNX Runtime CPU latency for model_fp32.onnx: 6.7 ms median on 1 thread. For predictable latency, use 1–4 threads: 16 threads lowered the median to 2.3 ms but raised the p95 to 12.8 ms.

How to use

Input and output

  • Input name: image, a float32 tensor of shape [N, 3, 300, 300].
  • Preprocessing: convert to 3-channel RGB (the source images are grayscale stored as RGB), resize to 300 Γ— 300, scale to [0, 1], then normalise with ImageNet mean [0.485, 0.456, 0.406] and std [0.229, 0.224, 0.225].
  • Output name: logits, shape [N, 2]. Apply softmax for confidence.
  • Class index 0 = defective (def_front in the source data). Class index 1 = acceptable (ok_front).

Example (ONNX Runtime)

import numpy as np
import onnxruntime as ort
from PIL import Image

session = ort.InferenceSession("model_fp32.onnx", providers=["CPUExecutionProvider"])

img = Image.open("part.jpeg").convert("RGB").resize((300, 300))
x = np.asarray(img, dtype=np.float32) / 255.0
x = (x - [0.485, 0.456, 0.406]) / [0.229, 0.224, 0.225]
x = x.transpose(2, 0, 1)[np.newaxis].astype(np.float32)

logits = session.run(["logits"], {"image": x})[0][0]
probs = np.exp(logits - logits.max())
probs /= probs.sum()

labels = ["defective", "acceptable"]
print(labels[int(probs.argmax())], float(probs.max()))

The source repository also includes a FastAPI service (POST /predict) that returns the predicted class, confidence, model version and inference time.

Training data and the leak-free split

Data: Kaggle, Real-Life Industrial Dataset of Casting Product (7,348 images, 300 Γ— 300, two classes).

Leakage found in the official split. Perceptual hashing showed that 70.1% of the official test images have a near-duplicate (Hamming distance ≀ 4) in the official training set, including 64 exact file duplicates. Visual review confirmed these are the same physical parts. Scores reported on the official split are therefore likely to overstate how well a model handles unseen parts.

The leak-free split used here:

  1. All 7,348 images were pooled and grouped by near-duplicate (union-find at Hamming distance ≀ 2), then split by group into train, validation and test (70 / 15 / 15, stratified by class).
  2. Any validation or test image that still had a training near-duplicate at distance ≀ 4 was excluded from evaluation, as were validation or test images involved in a label conflict.
  3. An independent from-scratch check found zero near-duplicate or exact-duplicate pairs across train, validation and test.
Split Defective Acceptable Total
Train 2,952 2,190 5,142
Validation 339 174 513
Test 366 182 548
Excluded 554 591 1,145

Label conflicts: 143 near-identical image pairs carry different labels. These conflicts are left unresolved. Conflicting images were kept in training and removed from validation and test.

Training procedure

  • Base model: torchvision MobileNetV3-Small, ImageNet-pretrained, with the classifier's final layer replaced by a 2-class output.
  • Stage A: backbone frozen; head trained with AdamW at lr 1e-3 for 5 epochs.
  • Stage B: all layers unfrozen; AdamW with head lr 1e-4 and backbone lr 1e-5, cosine decay, early stopping (patience 5), up to 25 epochs.
  • Selection: best validation macro-F1 (stage B, epoch 7).
  • Loss: cross-entropy. Batch size: 32. Seed: 42.
  • Augmentation (train only): rotation up to Β±10Β°, translation up to 5%, brightness and contrast jitter up to Β±10%. No flips, because a mirrored impeller is not a physically valid part.
  • Hardware: NVIDIA TITAN RTX.

Robustness and explainability checks

Background shortcut test (inference only, on the leak-free test set):

Input Accuracy Defect recall
Original image 99.82% 99.73%
Part only (background flattened) 99.27% 98.91%
Background only (part flattened) 49.82% 40.98%

The model's decisions come from the part itself. There is no usable signal in the background.

Grad-CAM: heatmaps are provided in the source repository. They come from a 10 Γ— 10 feature map, so they show roughly where the model looks but are too coarse to localise defects. Grad-CAM is an explanation technique, not proof of defect localisation.

Quantisation

Variant Size Defect recall False rejection rate Verdict
FP32 ONNX 5.8 MB 99.73% 0.00% βœ… Recommended
FP16 ONNX 2.9 MB 100%* 0.00% βœ… Compressed option
Static INT8 (best variant) 1.8 MB 79.5% 35.7% ❌ Not viable

* One borderline test image flipped to the correct class. Treat FP16 as equivalent to FP32, not better.

Static INT8 quantisation (ONNX Runtime QDQ, calibrated on training images only) degraded this model badly at every setting tried, including excluding the squeeze-and-excitation blocks. INT8 is not recommended for this model.

Limitations

  • Not validated on real factory images. All results come from one public dataset. A shadow-mode pilot on independently collected images from a real production line is required before any production use.
  • Narrow scope: one part type, one viewpoint, one lighting setup. Performance on other parts, cameras or conditions is unknown.
  • Small error counts: 1 missed defect in 366. Differences of one or two images between models are within run-to-run variation. Each model was trained with a single seed.
  • Unresolved label conflicts (143 near-identical pairs with different labels) mean some ground-truth labels in the source data are likely wrong.
  • Licence: the training data is licensed CC BY-NC-ND 4.0, so this model may not be used commercially. A commercial deployment requires retraining on data you have the rights to use.

Intended use

  • Demonstrating that a compact model can classify casting defects accurately at edge-device speed.
  • A starting point for fine-tuning on a manufacturer's own images, followed by proper validation.
  • Research and education on leakage-aware evaluation of industrial vision datasets.

Out of scope: making unsupervised accept/reject decisions on a production line without human oversight and site-specific validation, and any commercial use.

Licence and permitted use

This model is released under CC BY-NC-ND 4.0, matching the licence of its training data.

  • Allowed: downloading and running the model for non-commercial research, education, evaluation and demonstration, with attribution.
  • Not allowed: commercial use of any kind, including in products, paid services or client deliverables.
  • Not allowed: redistributing modified versions of the model.

Whether a trained model counts as a derivative of its training images is not legally settled. This licence takes the cautious reading. For commercial use, retrain the architecture and pipeline on images you have the rights to use.

Citation of data source

Dabhi, R. (2020). Real-Life Industrial Dataset of Casting Product. Kaggle. https://www.kaggle.com/datasets/ravirajsinh45/real-life-industrial-dataset-of-casting-product. Licensed under CC BY-NC-ND 4.0.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support