| --- |
| license: mit |
| datasets: |
| - gbharti/finance-alpaca |
| - lavita/ChatDoctor-HealthCareMagic-100k |
| - laion/OIG |
| - openai/webgpt_comparisons |
| - taskydata/GPT4Tools |
| - DataProvenanceInitiative/cot_submix_original |
| - 0x70DA/stackoverflow-chat-data |
| language: |
| - en |
| library_name: adapter-transformers |
| pipeline_tag: text-classification |
| --- |
| |
| # Attempt to reproduce Mixture-of-LoRAs classifier |
|
|
| Mixture-of-LoRAs: An Efficient Multitask Tuning for Large Language Models |
|
|
| https://arxiv.org/pdf/2403.03432 |
|
|
| ## Datasets |
|
|
| We evenly sample about 10k training data and 2k validation data on each dataset. |
|
|
| From `laion/OIG` was taken only: |
| - unified_merged_code_xp3.jsonl |
| - unified_grade_school_math_instructions.jsonl |
| - unified_mathqa_flanv2_kojma_cot.jsonl |
