File size: 3,174 Bytes
0599d19
 
 
 
 
 
 
 
 
 
 
006e5e9
0599d19
006e5e9
0599d19
 
 
006e5e9
0599d19
006e5e9
0599d19
 
 
 
006e5e9
0599d19
006e5e9
0599d19
414cfeb
006e5e9
0599d19
006e5e9
0599d19
 
 
 
 
 
006e5e9
0599d19
 
 
006e5e9
0599d19
006e5e9
0599d19
006e5e9
0599d19
 
69bfdab
 
 
 
 
 
006e5e9
0599d19
006e5e9
0599d19
 
69bfdab
006e5e9
0599d19
006e5e9
414cfeb
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
---

title: Computer Vision Classification Comparison
emoji: 📷
colorFrom: yellow
colorTo: blue
sdk: gradio
sdk_version: "6.12.0"
app_file: app.py
pinned: false
---

# Gradio Pokemon Classification App

This app compares 3 image classification approaches on custom Pokemon images:

- Fine-tuned transfer learning model (ResNet18) from this project
- Zero-shot CLIP (`openai/clip-vit-base-patch32`)
- OpenAI vision model (LLM image classification)

## Dataset Used For Training

- Custom dataset from course materials (week 8 style): `data/pokemon`
- Train split: `data/pokemon/train`
- Test split: `data/pokemon/test`
- Number of classes: `6` (`charizard`, `charmander`, `charmeleon`, `ditto`, `eevee`, `ekans`)

## Trained Model

- Local model artifact: `models/custom_resnet18.pth`
- Hugging Face model link: `https://huggingface.co/kukalend/pokemon-transfer-resnet18`

## Training Performance

| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|---:|---:|---:|---:|---:|
| 1.6525 | 1 | - | 0.4324 | 0.2848 |
| 0.9985 | 2 | - | 0.6216 | 0.7285 |
| 0.6668 | 3 | - | 0.7838 | 0.8344 |
| 0.4450 | 4 | - | 0.8378 | 0.9007 |

Notes:
- Validation loss was not logged separately in the training script, so the table reports validation accuracy in the validation column context and train accuracy in the accuracy column.
- Final test accuracy of the custom model: `0.80`.

## Example Image Results

The table below reports the true class and Top-3 predictions for Custom ResNet and CLIP.

| Image | True Class | Custom ResNet Top-3 (score) | CLIP Top-3 (score) | OpenAI LLM (label, confidence) |
|---|---|---|---|---|
| `charizard.png` | `charizard` | `eevee` (0.5464)<br>`charizard` (0.3056)<br>`charmander` (0.0711) | `charizard` (0.4536)<br>`charmander` (0.3524)<br>`charmeleon` (0.0896) | `charizard` (0.9000) |
| `charmander.png` | `charmander` | `charmander` (0.5801)<br>`charmeleon` (0.3410)<br>`ekans` (0.0315) | `charmeleon` (0.5400)<br>`charmander` (0.4202)<br>`charizard` (0.0268) | `charmander` (0.9500) |
| `charmeleon.png` | `charmeleon` | `charmeleon` (0.3503)<br>`eevee` (0.3164)<br>`ekans` (0.2516) | `charmeleon` (0.4247)<br>`eevee` (0.3977)<br>`charizard` (0.0802) | `charmeleon` (0.9500) |
| `ditto.png` | `ditto` | `ditto` (0.8271)<br>`ekans` (0.0959)<br>`eevee` (0.0370) | `ditto` (0.5273)<br>`ekans` (0.2055)<br>`eevee` (0.1169) | `ekans` (0.9000) |
| `eevee.png` | `eevee` | `eevee` (0.9962)<br>`ekans` (0.0022)<br>`charizard` (0.0006) | `eevee` (0.9984)<br>`ditto` (0.0008)<br>`charizard` (0.0004) | `eevee` (0.9500) |
| `ekans.png` | `ekans` | `ekans` (0.5366)<br>`eevee` (0.3374)<br>`charmander` (0.0682) | `ekans` (0.5120)<br>`charmeleon` (0.2303)<br>`charmander` (0.2177) | `ekans` (0.9500) |

## Overall Comparison (Test Set)

- Custom transfer learning model accuracy: `0.80`
- CLIP zero-shot model accuracy: `0.72`
- OpenAI vision model accuracy: `0.7083` on 24 evaluated samples

## Links

- Hugging Face Space: `https://huggingface.co/spaces/kukalend/computer-Vision-classification`
- Hugging Face model repo: `https://huggingface.co/kukalend/pokemon-transfer-resnet18`