skin-detection / API_DOCUMENTATION.md
refaat9900's picture
Upload API_DOCUMENTATION.md
fdfcef8 verified
|
Raw History Blame Contribute Delete
19.1 kB
# ๐Ÿฉบ Skin Disease Classifier โ€” API Documentation (v2.0 โ€” Dynamic AI Recommendations)
> ResNet50 (HAM10000) for classification + Groq (fast LLM inference) for personalized, dynamic risks & recommendations ยท Deployed on Hugging Face Spaces
![PyTorch](https://img.shields.io/badge/PyTorch-EE4C2C?style=flat&logo=pytorch&logoColor=white)
![FastAPI](https://img.shields.io/badge/FastAPI-009688?style=flat&logo=fastapi&logoColor=white)
![Groq](https://img.shields.io/badge/Groq-LLM-F55036?style=flat)
![HuggingFace](https://img.shields.io/badge/HuggingFace-FFD21E?style=flat&logo=huggingface&logoColor=black)
---
## ๐Ÿ†• What's new in v2.0
This version replaces the old **static** recommendations with a **two-step, dynamic** flow:
1. **`POST /predict`** โ€” classifies the image and returns a **set of follow-up questions specific to the predicted class** (e.g. Melanoma gets different questions than a Mole).
2. **`POST /recommend`** โ€” takes the predicted class + the user's answers to those questions, sends them to **Groq**, and returns **personalized risks & recommendations in both English and Arabic**, plus an `urgency_level`.
This means recommendations are no longer hardcoded โ€” they adapt to each patient's actual situation (how long the lesion has been there, whether it's bleeding, family history, etc).
---
## ๐Ÿ“Š Model Overview (Classifier)
| Property | Value |
|---|---|
| Architecture | ResNet50 (pretrained on ImageNet, fine-tuned) |
| Final layer | Linear(2048 โ†’ 7) |
| Input size | 224 ร— 224 ร— 3 (RGB) |
| Normalization | mean=[0.485, 0.456, 0.406] ยท std=[0.229, 0.224, 0.225] |
| Dataset | HAM10000 โ€” oversampled to 6,000 samples/class |
| Saved weights | `skin_resnet50.pth` |
| Recommendation engine | Groq โ€” model `llama-3.3-70b-versatile` |
---
## ๐Ÿ”— Base URL
```
https://refaat9900-skin-detection.hf.space
```
Swagger UI:
```
https://refaat9900-skin-detection.hf.space/docs
```
A working **demo web page** (`index.html`) that calls this exact API end-to-end โ€” upload โ†’ predict โ†’ answer questions โ†’ recommend โ€” is included alongside this documentation. Open it in any browser to see the full flow live.
---
## ๐Ÿ”‘ Required Server Configuration
The server needs a **Groq API key** set as an environment variable named `GROK_API_KEY`.
> โš ๏ธ Despite the variable name `GROK_API_KEY` (kept for backward compatibility), this must be a key from **Groq** (console.groq.com) โ€” **not** xAI's Grok (console.x.ai). These are two different companies and the keys are not interchangeable.
### On Hugging Face Spaces:
1. Go to your Space โ†’ **Settings** โ†’ **Variables and secrets**
2. Click **New secret**
3. Name: `GROK_API_KEY`
4. Value: *(your key from [console.groq.com/keys](https://console.groq.com/keys))*
5. Save โ€” the Space will restart automatically
> โš ๏ธ Never hardcode the API key in `main.py` or commit it to a public repo. Always use Space secrets / environment variables.
---
## ๐Ÿ“Œ Endpoints
| Method | Path | Description |
|---|---|---|
| `GET` | `/` | Health check |
| `POST` | `/predict` | Upload image โ†’ get class + confidence + **follow-up questions** |
| `POST` | `/recommend` | Send class + answers โ†’ get **AI-generated risks & recommendations** (EN + AR) |
---
## 1๏ธโƒฃ POST `/predict`
### Request
- `Content-Type: multipart/form-data`
- Field name: `file`
- Accepted: `image/jpeg`, `image/png`, `image/jpg`, `image/webp`
### Response
| Field | Type | Description |
|---|---|---|
| `predicted_class` | `string` | Predicted disease name (English) |
| `predicted_class_index` | `int` | Class index (0โ€“6) โ€” **save this, you need it for `/recommend`** |
| `confidence` | `float` | Confidence of top prediction (0โ€“100%) |
| `description` | `string` | General medical description (English) |
| `all_probabilities` | `object[]` | Confidence for all 7 classes, sorted descending |
| `followup_questions` | `object[]` | **Questions to ask the user**, specific to the predicted class |
### `followup_questions` item structure
| Field | Type | Description |
|---|---|---|
| `id` | `string` | Stable identifier โ€” **send this back exactly** in `/recommend` |
| `question_en` | `string` | Question text in English |
| `question_ar` | `string` | Question text in Arabic |
| `options` | `string[]` | Suggested answer choices (show as buttons/dropdown) |
### Example response
```json
{
"predicted_class": "Melanoma",
"predicted_class_index": 5,
"confidence": 94.32,
"description": "Melanoma is the most dangerous form of skin cancer...",
"all_probabilities": [
{ "class_index": 5, "class_name": "Melanoma", "confidence": 94.32 },
{ "class_index": 4, "class_name": "Melanocytic Nevi (Moles)", "confidence": 3.11 }
],
"followup_questions": [
{
"id": "duration",
"question_en": "How long have you noticed this spot or change?",
"question_ar": "ุงู„ุจู‚ุนุฉ ุฃูˆ ุงู„ุชุบูŠูŠุฑ ุฏู‡ ู…ูˆุฌูˆุฏ ู…ู† ู‚ุฏ ุฅูŠู‡ุŸ",
"options": ["Less than 1 month", "1โ€“6 months", "More than 6 months"]
},
{
"id": "abcde_change",
"question_en": "Has it changed in shape, border, color, or size recently (ABCDE rule)?",
"question_ar": "ุญุตู„ ุชุบูŠูŠุฑ ููŠ ุงู„ุดูƒู„ ุฃูˆ ุงู„ุญุฏูˆุฏ ุฃูˆ ุงู„ู„ูˆู† ุฃูˆ ุงู„ุญุฌู… ู…ุคุฎุฑู‹ุงุŸ",
"options": ["Yes, significantly", "Slightly", "No"]
},
{
"id": "bleeding_itching",
"question_en": "Is it bleeding, itching, or crusting?",
"question_ar": "ุจูŠู†ุฒู ุฃูˆ ุจูŠุญูƒูƒ ุฃูˆ ููŠู‡ ู‚ุดูˆุฑ ุนู„ูŠู‡ุŸ",
"options": ["Yes", "No"]
},
{
"id": "family_history",
"question_en": "Any personal or family history of melanoma?",
"question_ar": "ููŠ ุชุงุฑูŠุฎ ุดุฎุตูŠ ุฃูˆ ุนุงุฆู„ูŠ ู„ู„ู…ูŠู„ุงู†ูˆู…ุงุŸ",
"options": ["Yes", "No", "Not sure"]
},
{
"id": "sun_history",
"question_en": "History of severe sunburns or frequent tanning bed use?",
"question_ar": "ููŠ ุชุงุฑูŠุฎ ู„ุญุฑูˆู‚ ุดู…ุณ ุดุฏูŠุฏุฉ ุฃูˆ ุงุณุชุฎุฏุงู… ู…ุชูƒุฑุฑ ู„ุฃุฌู‡ุฒุฉ ุงู„ุชุณู…ูŠุฑุŸ",
"options": ["Yes", "No"]
}
]
}
```
> ๐Ÿ’ก Each of the 7 classes has its **own set of 4โ€“5 tailored questions** (Melanoma's questions differ from Mole's, which differ from Dermatofibroma's, etc).
---
## 2๏ธโƒฃ POST `/recommend`
Call this **after** the user answers the `followup_questions` from `/predict`.
### Request body (JSON)
```json
{
"predicted_class_index": 5,
"answers": {
"duration": "1โ€“6 months",
"abcde_change": "Yes, significantly",
"bleeding_itching": "Yes",
"family_history": "No",
"sun_history": "Yes"
}
}
```
| Field | Type | Required | Description |
|---|---|---|---|
| `predicted_class_index` | `int` | โœ… Yes | The index returned by `/predict` |
| `answers` | `object` | โœ… Yes | Key = question `id` from `/predict`, Value = the option the user picked |
> โš ๏ธ If you skip a question, just omit its key โ€” the backend will mark it as "Not answered" when building the AI prompt.
### Response
| Field | Type | Description |
|---|---|---|
| `predicted_class` | `string` | Class name (English) |
| `risks_en` | `string[]` | 3โ€“5 personalized risk points (English) |
| `risks_ar` | `string[]` | Same risks in Arabic |
| `recommendations_en` | `string[]` | 4โ€“6 personalized action recommendations (English) |
| `recommendations_ar` | `string[]` | Same recommendations in Arabic |
| `urgency_level` | `string` | One of: `"low"`, `"moderate"`, `"high"`, `"urgent"` |
### Example response
```json
{
"predicted_class": "Melanoma",
"risks_en": [
"Recent changes in shape, color, and size strongly increase concern for malignant melanoma",
"Bleeding or crusting can indicate active tissue change requiring urgent biopsy",
"History of severe sunburns is a known major risk factor for melanoma",
"Melanoma can spread quickly to lymph nodes if not treated early"
],
"risks_ar": [
"ุงู„ุชุบูŠูŠุฑุงุช ุงู„ุฃุฎูŠุฑุฉ ููŠ ุงู„ุดูƒู„ ูˆุงู„ู„ูˆู† ูˆุงู„ุญุฌู… ุชุฒูŠุฏ ู…ู† ุงุญุชู…ุงู„ูŠุฉ ูˆุฌูˆุฏ ู…ูŠู„ุงู†ูˆู…ุง ุฎุจูŠุซุฉ",
"ุงู„ู†ุฒูŠู ุฃูˆ ุงู„ู‚ุดูˆุฑ ู‚ุฏ ูŠุฏู„ ุนู„ู‰ ุชุบูŠุฑ ู†ุดุท ููŠ ุงู„ู†ุณูŠุฌ ูŠุญุชุงุฌ ุฎุฒุนุฉ ุนุงุฌู„ุฉ",
"ุชุงุฑูŠุฎ ุญุฑูˆู‚ ุงู„ุดู…ุณ ุงู„ุดุฏูŠุฏุฉ ุนุงู…ู„ ุฎุทุฑ ู…ุนุฑูˆู ู„ู„ู…ูŠู„ุงู†ูˆู…ุง",
"ุงู„ู…ูŠู„ุงู†ูˆู…ุง ู…ู…ูƒู† ุชู†ุชุดุฑ ุจุณุฑุนุฉ ู„ู„ุบุฏุฏ ุงู„ู„ูŠู…ูุงูˆูŠุฉ ู„ูˆ ู…ุงุชุนุงู„ุฌุชุด ุจุฏุฑูŠ"
],
"recommendations_en": [
"โš ๏ธ See a dermatologist or oncologist within the next few days โ€” do not delay",
"Avoid any further sun exposure or trauma to the area",
"Take a clear photo dated today to track any further changes",
"Ask your doctor about a biopsy given the recent ABCDE changes",
"Inform close family members to get their skin checked too"
],
"recommendations_ar": [
"โš ๏ธ ูŠุฌุจ ุฒูŠุงุฑุฉ ุทุจูŠุจ ุฌู„ุฏูŠุฉ ุฃูˆ ุฃูˆุฑุงู… ููŠ ุฃู‚ุฑุจ ูˆู‚ุช ู…ู…ูƒู† โ€” ู„ุง ุชุชุฃุฎุฑ",
"ุชุฌู†ุจ ุฃูŠ ุชุนุฑุถ ุฅุถุงููŠ ู„ู„ุดู…ุณ ุฃูˆ ุฅุตุงุจุฉ ููŠ ุงู„ู…ู†ุทู‚ุฉ",
"ุฎุฏ ุตูˆุฑุฉ ูˆุงุถุญุฉ ุจุชุงุฑูŠุฎ ุงู„ูŠูˆู… ู„ุชุชุงุจุน ุฃูŠ ุชุบูŠูŠุฑุงุช ุฅุถุงููŠุฉ",
"ุงุณุฃู„ ุงู„ุทุจูŠุจ ุนู† ุฅุฌุฑุงุก ุฎุฒุนุฉ ุจุณุจุจ ุงู„ุชุบูŠูŠุฑุงุช ุงู„ุฃุฎูŠุฑุฉ",
"ุฃุฎุจุฑ ุฃูุฑุงุฏ ุงู„ุนุงุฆู„ุฉ ุงู„ู…ู‚ุฑุจูŠู† ุจุนู…ู„ ูุญุต ู„ู„ุฌู„ุฏ ูƒุฐู„ูƒ"
],
"urgency_level": "urgent"
}
```
---
## ๐Ÿ”„ Full Flow Diagram
```
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” image โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚ Frontend โ”‚ โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–บ โ”‚ POST /predict โ”‚
โ”‚ (Flutter/JS)โ”‚ โ”‚ (ResNet50 model) โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
โ–ฒ โ”‚
โ”‚ class + confidence + โ”‚
โ”‚ followup_questions โ–ผ
โ”‚ Show questions to user
โ”‚ (buttons from `options`)
โ”‚ โ”‚
โ”‚ โ–ผ
โ”‚ User answers all/some
โ”‚ โ”‚
โ”‚ class_index + answers โ–ผ
โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ โ”‚ POST /recommend โ”‚
risks + recommendationsโ”‚ (calls Groq) โ”‚
(EN + AR) + urgency โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
```
---
## ๐Ÿท๏ธ Disease classes (label mapping)
| Index | Class name (EN) |
|---|---|
| 0 | Actinic Keratoses |
| 1 | Basal Cell Carcinoma |
| 2 | Benign Keratosis-like Lesions |
| 3 | Dermatofibroma |
| 4 | Melanocytic Nevi (Moles) |
| 5 | Melanoma |
| 6 | Vascular Lesions |
---
## ๐Ÿ’ป Integration Examples
### Flutter (Dart) โ€” Full flow
```dart
import 'dart:io';
import 'dart:convert';
import 'package:http/http.dart' as http;
const String baseUrl = 'https://refaat9900-skin-detection.hf.space';
// Step 1: Predict
Future<Map<String, dynamic>> predictSkinDisease(File imageFile) async {
final request = http.MultipartRequest('POST', Uri.parse('$baseUrl/predict'));
request.files.add(await http.MultipartFile.fromPath('file', imageFile.path));
final streamed = await request.send();
final response = await http.Response.fromStream(streamed);
if (response.statusCode != 200) {
throw Exception('Predict failed: ${response.statusCode} โ€” ${response.body}');
}
return jsonDecode(response.body) as Map<String, dynamic>;
}
// Step 2: Recommend (after user answers followup_questions)
Future<Map<String, dynamic>> getRecommendation(
int predictedClassIndex,
Map<String, String> answers,
) async {
final response = await http.post(
Uri.parse('$baseUrl/recommend'),
headers: {'Content-Type': 'application/json'},
body: jsonEncode({
'predicted_class_index': predictedClassIndex,
'answers': answers,
}),
);
if (response.statusCode != 200) {
throw Exception('Recommend failed: ${response.statusCode} โ€” ${response.body}');
}
return jsonDecode(response.body) as Map<String, dynamic>;
}
// Usage example
void example() async {
final file = File('/path/to/skin_image.jpg');
// 1. Predict
final prediction = await predictSkinDisease(file);
print('Predicted: ${prediction['predicted_class']}');
final classIndex = prediction['predicted_class_index'];
final questions = prediction['followup_questions'] as List;
// 2. Collect answers (in a real app, show these as UI โ€” e.g. buttons per `options`)
final Map<String, String> answers = {};
for (final q in questions) {
// Example: just pick the first option for demo purposes
answers[q['id']] = q['options'][0];
}
// 3. Get personalized recommendation
final result = await getRecommendation(classIndex, answers);
print('Urgency: ${result['urgency_level']}');
print('Risks (EN): ${result['risks_en']}');
print('Risks (AR): ${result['risks_ar']}');
print('Recommendations (EN): ${result['recommendations_en']}');
print('Recommendations (AR): ${result['recommendations_ar']}');
}
```
#### pubspec.yaml
```yaml
dependencies:
flutter:
sdk: flutter
http: ^1.2.1
image_picker: ^1.0.7
```
---
### JavaScript (Fetch) โ€” Full flow
```javascript
const BASE_URL = 'https://refaat9900-skin-detection.hf.space';
// Step 1: Predict
async function predictSkinDisease(imageFile) {
const formData = new FormData();
formData.append('file', imageFile);
const res = await fetch(`${BASE_URL}/predict`, { method: 'POST', body: formData });
if (!res.ok) throw new Error(`Predict failed: ${res.status}`);
return await res.json();
}
// Step 2: Recommend
async function getRecommendation(predictedClassIndex, answers) {
const res = await fetch(`${BASE_URL}/recommend`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ predicted_class_index: predictedClassIndex, answers }),
});
if (!res.ok) throw new Error(`Recommend failed: ${res.status}`);
return await res.json();
}
// Usage
const fileInput = document.querySelector('#upload');
fileInput.addEventListener('change', async (e) => {
const file = e.target.files[0];
const prediction = await predictSkinDisease(file);
// Render prediction.followup_questions as a form (radio buttons per `options`)
renderQuestionsForm(prediction.followup_questions, async (answers) => {
const result = await getRecommendation(prediction.predicted_class_index, answers);
console.log('Urgency:', result.urgency_level);
console.log('Risks (EN):', result.risks_en);
console.log('Recommendations (AR):', result.recommendations_ar);
});
});
```
---
### React โ€” Full flow component
```jsx
import { useState } from 'react';
const BASE_URL = 'https://refaat9900-skin-detection.hf.space';
export default function SkinChecker() {
const [prediction, setPrediction] = useState(null);
const [answers, setAnswers] = useState({});
const [recommendation, setRecommendation] = useState(null);
const [loading, setLoading] = useState(false);
const handleUpload = async (e) => {
const file = e.target.files[0];
if (!file) return;
setLoading(true);
const formData = new FormData();
formData.append('file', file);
const res = await fetch(`${BASE_URL}/predict`, { method: 'POST', body: formData });
const data = await res.json();
setPrediction(data);
setAnswers({});
setRecommendation(null);
setLoading(false);
};
const submitAnswers = async () => {
setLoading(true);
const res = await fetch(`${BASE_URL}/recommend`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
predicted_class_index: prediction.predicted_class_index,
answers,
}),
});
const data = await res.json();
setRecommendation(data);
setLoading(false);
};
return (
<div>
<input type="file" accept="image/jpeg,image/png,image/webp" onChange={handleUpload} />
{prediction && (
<div>
<h2>{prediction.predicted_class} โ€” {prediction.confidence}%</h2>
<p>{prediction.description}</p>
<h3>A few quick questions</h3>
{prediction.followup_questions.map((q) => (
<div key={q.id}>
<p>{q.question_en} / {q.question_ar}</p>
{q.options.map((opt) => (
<button
key={opt}
onClick={() => setAnswers((prev) => ({ ...prev, [q.id]: opt }))}
style={{ fontWeight: answers[q.id] === opt ? 'bold' : 'normal' }}
>
{opt}
</button>
))}
</div>
))}
<button onClick={submitAnswers} disabled={loading}>
{loading ? 'Analyzing...' : 'Get Recommendation'}
</button>
</div>
)}
{recommendation && (
<div>
<h3>Urgency: {recommendation.urgency_level}</h3>
<h4>Risks</h4>
<ul>{recommendation.risks_en.map((r, i) => <li key={i}>{r}</li>)}</ul>
<h4>ุงู„ุชูˆุตูŠุงุช</h4>
<ul>{recommendation.recommendations_ar.map((r, i) => <li key={i}>{r}</li>)}</ul>
</div>
)}
</div>
);
}
```
---
### cURL (terminal testing)
```bash
# Step 1: Predict
curl -X POST "https://refaat9900-skin-detection.hf.space/predict" \
-F "file=@skin_image.jpg"
# Step 2: Recommend (use predicted_class_index & question ids from step 1's response)
curl -X POST "https://refaat9900-skin-detection.hf.space/recommend" \
-H "Content-Type: application/json" \
-d '{
"predicted_class_index": 5,
"answers": {
"duration": "1โ€“6 months",
"abcde_change": "Yes, significantly",
"bleeding_itching": "Yes",
"family_history": "No",
"sun_history": "Yes"
}
}'
```
---
## โš ๏ธ Error Responses
| Endpoint | HTTP code | Cause | Fix |
|---|---|---|---|
| `/predict` | `400` | Invalid file type or corrupted image | Send valid JPEG/PNG/WebP |
| `/predict` | `422` | Missing `file` field | Field name must be exactly `file` |
| `/recommend` | `400` | Invalid `predicted_class_index` | Must be an integer 0โ€“6 |
| `/recommend` | `422` | Missing `predicted_class_index` or `answers` | Both fields are required in the JSON body |
| `/recommend` | `500` | `GROK_API_KEY` not set on server | Add it as a Space secret |
| `/recommend` | `502` | Groq API unreachable or returned a bad response | Check API key validity / Groq service status |
---
## ๐Ÿ“‹ Medical Disclaimer
> This API is intended for **research and educational purposes only**. The AI-generated risks and recommendations (including those from Groq) do **not** replace professional medical diagnosis. Always consult a qualified dermatologist for any skin-related health concerns.