hkondle commited on
Commit
7039cbd
·
verified ·
1 Parent(s): bb2798e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -14
README.md CHANGED
@@ -11,20 +11,7 @@ tags:
11
 
12
  # VisAble diagram captioner
13
 
14
- BLIP fine-tuned on AI2D-Caption diagrams to produce part-to-whole descriptions for blind and
15
- low-vision students. The vision encoder was frozen and only the text decoder was trained, using a
16
- cross entropy loss that up-weights AI2D entity labels by 5.0x.
17
 
18
  Trained for 5 epochs, batch size 8, learning rate 5e-05.
19
 
20
- ```python
21
- from transformers import BlipForConditionalGeneration, BlipProcessor
22
- from PIL import Image
23
-
24
- processor = BlipProcessor.from_pretrained("hkondle/CapstoneMainModel")
25
- model = BlipForConditionalGeneration.from_pretrained("hkondle/CapstoneMainModel")
26
-
27
- image = Image.open("diagram.png").convert("RGB")
28
- inputs = processor(images=image, return_tensors="pt")
29
- print(processor.decode(model.generate(**inputs, max_length=256, num_beams=4)[0], skip_special_tokens=True))
30
- ```
 
11
 
12
  # VisAble diagram captioner
13
 
14
+ BLIP fine-tuned on AI2D-Caption dataset to produce part-to-whole descriptions for blind students.
 
 
15
 
16
  Trained for 5 epochs, batch size 8, learning rate 5e-05.
17