File size: 2,691 Bytes
bd659a9 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 | # Fuse-MD API
This folder contains the FastAPI-based inference service for Fuse-MD.
## What it does
The API loads a saved Fuse-MD checkpoint and exposes HTTP endpoints for:
- health checks
- model metadata
- multimodal prediction from text and an uploaded image
## Endpoints
- `GET /`
- `GET /health`
- `GET /model-info`
- `POST /predict`
## Easy local inference
1. Activate your virtual environment.
2. Download the text backbone once:
```bash
python api/setup_local_model.py --model VishnuPJ/MalayaLLM_7B_Base
```
3. Open `api/local_config.py` and set `CHECKPOINT_PATH` to your trained `.pth` file.
4. Start the API from the repository root:
```bash
python api/run_api.py
```
5. In a second terminal, send a prediction request:
```bash
python api/predict.py --image data/malayalam/dev/sample/148.jpg --text "sample meme transcription"
```
The helper scripts use the values in `api/local_config.py` by default, so you
only need to set the checkpoint path once for the common local workflow.
The setup command requires internet only the first time. After that, runtime
loads the text backbone from `local_models/` in offline mode. If you use a
Tamil checkpoint, run
`python api/setup_local_model.py --model abhinand/tamil-llama-7b-base-v0.1`
once too.
## Local config defaults
`api/local_config.py` includes:
- `CHECKPOINT_PATH`
- `LOCAL_MODEL_ROOT`
- `HOST`
- `PORT`
- `DEVICE`
- `THRESHOLD`
- `MAX_LENGTH`
You can also override the startup settings directly from the terminal:
```bash
python api/run_api.py --checkpoint trained_model/tamil/fusion/your_checkpoint.pth --device cpu
```
The model setup command also supports checkpoint-based inference of the model id:
```bash
python api/setup_local_model.py --checkpoint trained_model/malayalam/fusion/your_checkpoint.pth
```
## Run the API manually
From the repository root:
```bash
uvicorn api.api.app:app --host 0.0.0.0 --port 8000 --reload
```
## Optional environment variables
- `FUSEMD_CHECKPOINT`
- `FUSEMD_HOST`
- `FUSEMD_PORT`
- `FUSEMD_THRESHOLD`
- `FUSEMD_MAX_LENGTH`
- `FUSEMD_USE_8BIT`
- `FUSEMD_DEVICE`
- `FUSEMD_LOCAL_MODEL_ROOT`
## Example request
```bash
python api/predict.py --image sample.jpg --text "sample meme transcription"
```
Manual `curl` usage still works:
```bash
curl -X POST "http://127.0.0.1:8000/predict" \
-F "text=sample meme transcription" \
-F "image=@sample.jpg"
```
## Notes
- The API uses the saved checkpoint metadata to rebuild the model.
- The repository does not include the dataset.
- The text backbone must be downloaded once into `local_models/` before offline runtime.
- For single-image API inference, uploaded images are normalized per image at runtime.
|