mjpsm's picture
Add system-prompt KV cache; prompt-only request; drop raw_output
bf18f01 verified
|
Raw History Blame Contribute Delete
1.34 kB
metadata
title: MyVillage Project Creation API V2
emoji: 🎓
colorFrom: blue
colorTo: purple
sdk: docker
app_port: 7860
models:
  - mjpsm/project-creation-qwen2.5-1.5b-sft-v2

MyVillage Project Creation API (V2)

FastAPI endpoint for the V2 fine-tuned project-creation model. Upload the merged model first. mjpsm/project-creation-qwen2.5-1.5b-sft-v2 must be public, or add HF_TOKEN as a Space secret.

POST /predict:

{"prompt": "Can you put together a project for 7th graders on ocean conservation?"}

Only prompt is accepted; generation settings are fixed server-side (max_new_tokens=512, temperature=0.7, top_p=0.9, repetition_penalty=1.08).

Returns title, description, paragraphs, and model. K–5 requests produce 4 paragraphs (including parents and caregivers); grade 6+ requests produce 3.

Operational notes

  • The system prompt's KV cache is computed once at startup and copied per request, so each request only prefills the user prompt.
  • TORCH_NUM_THREADS defaults to 2 (CPU Basic's vCPUs). Raise it if you upgrade the hardware.
  • A public Space is a public endpoint; add authentication/rate limiting before production use.
  • This is a separate Space from V1; switch MyVillage over only after comparing outputs.
  • Set MODEL_ID in Space Variables if you change the model repository.