Update README.md

README.md

@@ -26,14 +26,34 @@ from transformers import AutoTokenizer, AutoModelForCausalLM
 tokenizer = AutoTokenizer.from_pretrained("Cedille/de-anna")
 model = AutoModelForCausalLM.from_pretrained("Cedille/de-anna")
 ```
-### Lower memory usage (loads on 16GB of RAM)
-GPT_J models (link) have a parameter to only be loaded once ...
-Combine that with half precision (fp16) ...
-TO DO
+### Lower memory usage
+Loading a model with Huggingface requires 2 copies of the weights, so 48+ GB of RAM for [GPT_J models](https://huggingface.co/docs/transformers/v4.15.0/model_doc/gptj) in float32 precision.
+The first trick would be to load the model with the specific argument below to load only 1 copy of the weights.
+```
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+tokenizer = AutoTokenizer.from_pretrained("Cedille/de-anna")
+model = AutoModelForCausalLM.from_pretrained("Cedille/de-anna", low_cup_mem_usage=True)
+```
+
+We are planning on adding an fp16 branch soon. Combined with the lower memory loading above, loading could be done on 12.1GB of RAM.
 
-## Generation
-TO DO
+## Generation example
+```
+model.eval()
+input_sentence = "Personne ne me connaissait à Buckton. Clem avait choisi la ville à cause de cela"
+input_ids = tokenizer.encode(input_sentence, return_tensors='pt')
 
+beam_outputs = model.generate(
+    input_ids, 
+    max_length=100, 
+    do_sample=True,   
+    top_k=50, 
+    top_p=0.95, 
+    num_return_sequences=1
+)
+print(tokenizer.decode(beam_outputs[0], skip_special_tokens=True))
+```
 ## Contact us
 For any custom development please contact us at hello@cedille.ai.