| --- |
| language: en |
| tags: |
| - arxiv |
| - research-papers |
| - text-generation |
| license: apache-2.0 |
| --- |
| |
| # KnullAI v2 - Fine-tuned on ArXiver Dataset |
|
|
| This model is a fine-tuned version of KnullAI v2, specifically trained on the ArXiver dataset containing research paper information. |
|
|
| ## Training Data |
| The model was fine-tuned on the neuralwork/arxiver dataset, which contains: |
| - Paper titles |
| - Abstracts |
| - Authors |
| - Publication dates |
| - Links |
|
|
| ## Model Details |
| - Base model: Rawkney/knullAi_v2 |
| - Training type: Causal language modeling |
| - Hardware: T4 GPU |
| - Mixed precision: FP16 |
| |
| ## Usage |
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| # Load model and tokenizer |
| model = AutoModelForCausalLM.from_pretrained("YOUR_REPO_ID") |
| tokenizer = AutoTokenizer.from_pretrained("YOUR_REPO_ID") |
| |
| # Example usage |
| title = "Your paper title" |
| input_text = f"Title: {title}\nAbstract:" |
| inputs = tokenizer(input_text, return_tensors="pt").to("cuda") |
|
|
| outputs = model.generate( |
| inputs["input_ids"], |
| max_length=256, |
| temperature=0.7, |
| top_p=0.9, |
| pad_token_id=tokenizer.eos_token_id |
| ) |
| |
| response = tokenizer.decode(outputs[0], skip_special_tokens=True) |
| print(response) |
| ``` |
| |
| ## Training Parameters |
| - Learning rate: 1e-5 |
| - Epochs: 1 |
| - Batch size: 1 |
| - Gradient accumulation steps: 16 |
| - Mixed precision training (fp16) |
| - Max sequence length: 512 |
| |
