| --- |
| license: apache-2.0 |
| --- |
| |
| This repo offers a set of fairseq roberta based models fine tuned to specific NLP tasks. |
| These models are: |
| - Small |
| - Easy to host on T4 or V100 |
| - 100x faster than using LLMs for similar tasks |
| - Easy to fine tune |
|
|
| All the models below were trained at Nlmatics Corp. from 2019-2023 with base model from: https://github.com/facebookresearch/fairseq/blob/main/examples/roberta/README.md |
|
|
| ### To run the models: |
| Use https://github.com/nlmatics/nlm-model-service |
|
|
| ### To acccess the models |
| Use https://github.com/nlmatics/nlm-utils |
|
|
| ### To train the models |
| TBD |
|
|
| ## List of Models |
|
|
| Click on each model to see details: |
|
|
| ### roberta.large.boolq |
|
|
| *Location:* [roberta.large.boolq](https://huggingface.co/ansukla/roberta/tree/main/roberta.large.boolq) |
|
|
| Trained with MNLI + Boolq |
|
|
| *Trained by:* Evan Li |
|
|
| *Application:* Given a passage and a question, answer the question with yes, no or unsure. |
|
|
| *Training Process:* https://blogs.nlmatics.com/2020/03/12/Boolean-Question-Answering-with-Neutral-Labels.html |
|
|
| ### roberta.large.qa |
| See folder: [roberta.large.qa](https://huggingface.co/ansukla/roberta/tree/main/roberta.large.qa) |
|
|
| Trained with SQuAD 2.0 + Custom Dataset preferring shorter spans better suited for data extraction |
|
|
| *Trained by:* Ambika Sukla |
|
|
| *Application:* Given a passage and a question, pick the shortest span from the passage that answers the question |
|
|
| *Training Process:* start, end location head on the top of Roberta Base |
|
|
| ### roberta.large.stsb |
| *See folder:* [roberta.large.stsb](https://huggingface.co/ansukla/roberta/tree/main/roberta.large.stsb) |
|
|
| Trained with STSB dataset |
|
|
| *Trained by:* Meta/Fairseq |
|
|
| *Application:* Given two passages, return a score beteen 0 and 1 to evaluate their similarity |
|
|
| *Training Process:* regression head on top of Roberta Base |
|
|
| ### roberta.large.phraseqa |
| *See folder:* [roberta.large.phraseqa](https://huggingface.co/ansukla/roberta/tree/main/roberta.large.phraseqa) |
|
|
| Trained with Roberta 2.0 with the question words removed from the question |
|
|
| *Trained By:* Batya Stein |
|
|
| *Application:* Given a passage and phrase (key), extract a value from the passage |
|
|
| *Training Process:* https://blogs.nlmatics.com/2020/08/25/Optimizing-Transformer-Q&A-Models-for-Naturalistic-Search.html |
|
|
| ### roberta.large.qasrl |
|
|
| *See folder:* [roberta.large.qasrl](https://huggingface.co/ansukla/roberta/tree/main/roberta.large.qasrl) |
|
|
| Trained with QASRL dataset |
|
|
| *Application:* Given a passage, get back values for who, what, when, where etc. |
|
|
| *Trained By:* Nima Sheikholeslami |
|
|
| ### roberta.large.qatype.lower.RothWithQ |
|
|
| *See folder:* [roberta.large.qatype.lower.RothWithQ](https://huggingface.co/ansukla/roberta/tree/main/roberta.large.qatype.lower.RothWithQ) |
|
|
| Trained with the Roth Question Type dataset. |
|
|
| *Application:* Given a question, return one of the answer types e.g. number, location. See the Roth dataset for full list. |
|
|
| *Trained By:* Evan Li |
|
|
| ### roberta.large.io_qa |
| |
| See folder: [roberta.large.io_qa](https://huggingface.co/ansukla/roberta/tree/main/roberta.large.io_qa) |
| Trained with SQuAD 2.0 dataset |
| |
| *Trained By:* Nima Sheikholeslami |
| |
| *Training Process:* Use io head to support multiple spans. |
| |
| |
