api/README.md

Fuse-MD API

This folder contains the FastAPI-based inference service for Fuse-MD.

What it does

The API loads a saved Fuse-MD checkpoint and exposes HTTP endpoints for:

• health checks
• model metadata
• multimodal prediction from text and an uploaded image

Endpoints

• GET /
• GET /health
• GET /model-info
• POST /predict

Easy local inference

1. Activate your virtual environment.
2. Download the text backbone once:

python api/setup_local_model.py --model VishnuPJ/MalayaLLM_7B_Base

3. Open api/local_config.py and set CHECKPOINT_PATH to your trained .pth file.
4. Start the API from the repository root:

python api/run_api.py

5. In a second terminal, send a prediction request:

python api/predict.py --image data/malayalam/dev/sample/148.jpg --text "sample meme transcription"

The helper scripts use the values in api/local_config.py by default, so you only need to set the checkpoint path once for the common local workflow.

The setup command requires internet only the first time. After that, runtime loads the text backbone from local_models/ in offline mode. If you use a Tamil checkpoint, run python api/setup_local_model.py --model abhinand/tamil-llama-7b-base-v0.1 once too.

Local config defaults

api/local_config.py includes:

• CHECKPOINT_PATH
• LOCAL_MODEL_ROOT
• HOST
• PORT
• DEVICE
• THRESHOLD
• MAX_LENGTH

You can also override the startup settings directly from the terminal:

python api/run_api.py --checkpoint trained_model/tamil/fusion/your_checkpoint.pth --device cpu

The model setup command also supports checkpoint-based inference of the model id:

python api/setup_local_model.py --checkpoint trained_model/malayalam/fusion/your_checkpoint.pth

Run the API manually

From the repository root:

uvicorn api.api.app:app --host 0.0.0.0 --port 8000 --reload

Optional environment variables

• FUSEMD_CHECKPOINT
• FUSEMD_HOST
• FUSEMD_PORT
• FUSEMD_THRESHOLD
• FUSEMD_MAX_LENGTH
• FUSEMD_USE_8BIT
• FUSEMD_DEVICE
• FUSEMD_LOCAL_MODEL_ROOT

Example request

python api/predict.py --image sample.jpg --text "sample meme transcription"

Manual curl usage still works:

curl -X POST "http://127.0.0.1:8000/predict" \
  -F "text=sample meme transcription" \
  -F "image=@sample.jpg"

Notes

• The API uses the saved checkpoint metadata to rebuild the model.
• The repository does not include the dataset.
• The text backbone must be downloaded once into local_models/ before offline runtime.
• For single-image API inference, uploaded images are normalized per image at runtime.