| license: cc-by-4.0 | |
| This model judges if a given output is sufficient to recreate a given instruction. | |
| It's useful for filtering data to train a reverse instruct model. It could also have applications around determining if an output/instruction pair is linked, or around quality filtering data (data where the instruction can be recreated from the output might be higher quality). | |
| The model is a binary classifier trained on top of Python 410m with 100k examples for 1 epoch. The final validation loss is .35. You can see an example of a dataset filtered with this model [here](https://huggingface.co/datasets/vikp/reverse_instruct). | |
| To use it, pass in this prompt format: | |
| ``` | |
| Output | |
| {output} | |
| Instruction | |
| {instruction} | |
| ``` | |
| Output should be the output from a model, and instruction should be the instruction that generated the output. The model will return a 0-1 score indicating how effectively the instruction can be recreated. |