| --- |
| base_model: unsloth/gemma-3n-e2b-it-unsloth-bnb-4bit |
| tags: |
| - text-generation-inference |
| - transformers |
| - unsloth |
| - gemma3n |
| - trl |
| language: |
| - en |
| license: apache-2.0 |
| datasets: |
| - Srinivasmec26/Educational-Flashcards-for-Global-Learners |
| - Srinivasmec26/Task-Reminders-for-Education-and-Professionals |
| - Srinivasmec26/Multidisciplinary-Educational-Summaries |
| - Srinivasmec26/Structured-Todo-Lists-for-Learning-and-Projects |
| library_name: adapter-transformers |
| --- |
| |
| # MindSlate: Fine-tuned Gemma-3B for Personal Knowledge Management |
|
|
| [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="250"/>](https://github.com/unslothai/unsloth) |
|
|
| ## Model Description |
|
|
| **MindSlate** is a fine-tuned version of Google's Gemma-3B model, optimized for personal knowledge management tasks including flashcard generation, reminder processing, content summarization, and task management. The model was trained using Unsloth's efficient fine-tuning techniques for 2x faster training. |
|
|
| - **Architecture**: Gemma-3B with LoRA adapters |
| - **Model type**: Causal Language Model |
| - **Fine-tuning method**: 4-bit QLoRA |
| - **Languages**: English |
| - **License**: Apache 2.0 |
| - **Developed by**: [Srinivas Nampalli](https://www.linkedin.com/in/srinivas-nampalli/) |
| - **Finetuned from**: [unsloth/gemma-3b-E2B-it-unsloth-bnb-4bit](https://huggingface.co/unsloth/gemma-3b-E2B-it-unsloth-bnb-4bit) |
|
|
| ## Model Sources |
|
|
| - **Repository**: [https://github.com/Srinivasmec26/MindSlate](https://github.com/Srinivasmec26/MindSlate) |
| - **Base Model**: [unsloth/gemma-3b-E2B-it-unsloth-bnb-4bit](https://huggingface.co/unsloth/gemma-3b-E2B-it-unsloth-bnb-4bit) |
|
|
| ## Uses |
|
|
| ### Direct Use |
| MindSlate is designed for: |
| - Automatic flashcard generation from study materials |
| - Intelligent reminder creation |
| - Content summarization |
| - Task extraction and organization |
| - Personal knowledge base management |
|
|
| ### Downstream Use |
| Can be integrated into: |
| - Educational platforms |
| - Productivity apps |
| - Note-taking applications |
| - Personal AI assistants |
|
|
| ### Out-of-Scope Use |
| Not suitable for: |
| - Medical or legal advice |
| - High-stakes decision making |
| - Generating factual content without verification |
|
|
| ## How to Get Started |
|
|
| ```python |
| from unsloth import FastLanguageModel |
| import torch |
| |
| # Load model with Unsloth optimizations |
| model, tokenizer = FastLanguageModel.from_pretrained( |
| model_name="Srinivasmec26/MindSlate", |
| max_seq_length=2048, |
| dtype=torch.float16, |
| load_in_4bit=True, |
| ) |
| |
| # Set chat template |
| tokenizer = FastLanguageModel.get_chat_template( |
| tokenizer, |
| chat_template="gemma", # Use "chatml" or other templates if needed |
| ) |
| |
| # Create prompt |
| messages = [ |
| {"role": "user", "content": "Convert to flashcard: Neural networks are computational models..."}, |
| ] |
| |
| # Generate response |
| inputs = tokenizer.apply_chat_template( |
| messages, |
| return_tensors="pt", |
| ).to("cuda") |
| |
| outputs = model.generate( |
| **inputs, |
| max_new_tokens=256, |
| temperature=0.7, |
| top_p=0.95, |
| ) |
| print(tokenizer.decode(outputs[0])) |
| ``` |
|
|
| ## Training Details |
|
|
| ### Training Data |
| The model was fine-tuned on a combination of structured datasets: |
|
|
| 1. **Flashcards Dataset** (400 items): |
| ```bibtex |
| @misc{educational_flashcards_2025, |
| title = {Multicultural Educational Flashcards Dataset}, |
| author = {Srinivas, Yathi Pachauri, Swarnim Gupta}, |
| year = {2025}, |
| publisher = {Hugging Face}, |
| url = {https://huggingface.co/datasets/Srinivasmec26/Educational-Flashcards-for-Global-Learners} |
| } |
| |
| ``` |
|
|
| 2. **Reminders Dataset** (100 items): |
| - *Private collection of contextual reminders* |
| - Format: {"input": "Meeting with team", "output": {"time": "2025-08-15 14:00", "location": "Zoom"}} |
| ```bibtex |
| @misc{educational_flashcards_2025, |
| title = {Multicultural Educational Flashcards Dataset}, |
| author = {Srinivas, Yathi Pachauri, Swarnim Gupta}, |
| year = {2025}, |
| publisher = {Hugging Face}, |
| url = {https://huggingface.co/datasets/Srinivasmec26/Educational-Flashcards-for-Global-Learners} |
| } |
| |
| ``` |
|
|
| 3. **Summaries Dataset** (100 items): |
| - *Academic paper abstracts and summaries* |
| - Collected from arXiv and academic publications |
| ```bibtex |
| @misc{knowledge_summaries_2025, |
| title = {Multidisciplinary-Educational-Summaries}, |
| author = {Srinivas Nampalli, Yathi Pachauri, Swarnim Gupta}, |
| year = {2025}, |
| publisher = {Hugging Face}, |
| url = {https://huggingface.co/datasets/Srinivasmec26/Multidisciplinary-Educational-Summaries} |
| } |
| ``` |
|
|
| 4. **Todos Dataset** (100 items): |
| ```bibtex |
| @misc{academic_todos_2025, |
| title = {Structured To-Do Lists for Learning and Projects}, |
| author = {Nampalli Srinivas, Yathi Pachauri, Swarnim Gupta}, |
| year = {2025}, |
| publisher = {Hugging Face}, |
| version = {1.0}, |
| url = {https://huggingface.co/datasets/Srinivasmec26/Structured-Todo-Lists-for-Learning-and-Projects} |
| } |
| |
| ``` |
|
|
| ### Training Procedure |
| - **Preprocessing**: Standardized into `### Input: ... \n### Output: ...` format |
| - **Framework**: Unsloth 2025.8.1 + Hugging Face TRL |
| - **Hardware**: Tesla T4 GPU (16GB VRAM) |
| - **Training Time**: 51 minutes for 3 epochs |
| - **LoRA Configuration**: |
| ```python |
| r=64, # LoRA rank |
| lora_alpha=128, # LoRA scaling factor |
| target_modules=["q_proj", "k_proj", "v_proj", "o_proj", |
| "gate_proj", "up_proj", "down_proj"], |
| ``` |
| - **Optimizer**: AdamW 8-bit |
| - **Learning Rate**: 2e-4 with linear decay |
|
|
| ## Evaluation |
| *Comprehensive benchmark results will be uploaded in v1.1. Preliminary metrics:* |
|
|
| | Metric | Value | |
| |----------------------|--------| |
| | **Training Loss** | 0.1284 | |
| | **Perplexity** | TBD | |
| | **Task Accuracy** | TBD | |
| | **Inference Speed** | 42 tokens/sec (T4) | |
|
|
| ## Technical Specifications |
|
|
| | Parameter | Value | |
| |----------------------|---------------------| |
| | Model Size | 3B parameters | |
| | Quantization | 4-bit (bnb) | |
| | Max Sequence Length | 2048 tokens | |
| | Fine-tuned Params | 1.66% (91.6M) | |
| | Precision | BF16/FP16 mixed | |
| | Architecture | Transformer Decoder | |
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{mindslate2025, |
| author = {Srinivas Nampalli }, |
| title = {MindSlate: Efficient Personal Knowledge Management with Gemma-3B}, |
| year = {2025}, |
| publisher = {Hugging Face}, |
| howpublished = {\url{https://huggingface.co/Srinivasmec26/MindSlate}}, |
| note = {Fine-tuned using Unsloth for efficient training} |
| } |
| ``` |
|
|
| ## Acknowledgements |
| - [Unsloth](https://github.com/unslothai/unsloth) for 2x faster fine-tuning |
| - Google for the [Gemma 3n](https://huggingface.co/sparkreaderapp/gemma-3n-E2B-it) base model |
| - Hugging Face for [TRL](https://huggingface.co/docs/trl) library |
|
|
| ## Model Card Contact |
| For questions and collaborations: |
| - Srinivas Nampalli: [LinkedIn](https://www.linkedin.com/in/srinivas-nampalli/) |
