Upload the checkpoint that produces the good descriptions

README.md

@@ -11,7 +11,20 @@ tags:
 
 # VisAble diagram captioner
 
-BLIP fine-tuned on AI2D-Caption dataset to produce part-to-whole descriptions for blind students. 
+BLIP fine-tuned on AI2D-Caption diagrams to produce part-to-whole descriptions for blind and
+low-vision students. The vision encoder was frozen and only the text decoder was trained, using a
+cross entropy loss that up-weights AI2D entity labels by 5.0x.
 
-Trained for 5 epochs, batch size 8, learning rate 5e-05.
+Trained for 10 epochs, batch size 8, learning rate 5e-05.
 
+```python
+from transformers import BlipForConditionalGeneration, BlipProcessor
+from PIL import Image
+
+processor = BlipProcessor.from_pretrained("hkondle/CapstoneMainModel-10ep")
+model = BlipForConditionalGeneration.from_pretrained("hkondle/CapstoneMainModel-10ep")
+
+image = Image.open("diagram.png").convert("RGB")
+inputs = processor(images=image, return_tensors="pt")
+print(processor.decode(model.generate(**inputs, max_length=256, num_beams=4)[0], skip_special_tokens=True))
+```

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:844a760f15d69feedd68c515b0bf69c29f57f56cfd88dd498ffaf02121e0c6dc
+oid sha256:2c079de1c52f8650f54c7b368e66de66bd0d1cf0bab76b0087f6c409d7aea8c0
 size 989839224