| --- |
| language: multilingual |
| license: mit |
| tags: |
| - transformer |
| - summarization |
| - translation |
| - question-answering |
| - english |
| - arabic |
| datasets: |
| - miscovery/arabic_egypt_english_world_facts |
| pipeline_tag: summarization |
| library_name: transformers |
| --- |
| |
| # Miscovery Transformer Model |
|
|
| This model is a transformer-based encoder-decoder model for multiple NLP tasks: |
| - Text summarization |
| - Translation (English-Arabic) |
| - Question-answering |
|
|
| ## Model Architecture |
|
|
| - Model type: miscovery |
| - Number of parameters: 485674144 |
| - Encoder layers: 12 |
| - Decoder layers: 12 |
| - Attention heads: 12 |
| - Hidden size: 768 |
| - Feed-forward size: 3072 |
|
|
| ## Training |
|
|
| The model was trained in two stages: |
| 1. Pre-training on sentence rearrangement tasks |
| 2. Fine-tuning on downstream tasks |
|
|
| ## Usage |
|
|
| 1. Install the package: |
|
|
| ```bash |
| pip install miscovery-model |
| ``` |
|
|
| 2. Run the model using a script: |
|
|
| ```python |
| from miscovery_model import standard_pipeline |
| |
| # Create a pipeline |
| model = standard_pipeline("miscovery/model") |
| |
| # Use it |
| result = model("Translate this to Arabic: What year did World War I begin?") |
| print(result) |
| ``` |
|
|
| ## Limitations |
|
|
| This model was trained on specific datasets and may not generalize well to all domains. |
