Update README

README.md

@@ -1,111 +1,88 @@
+---
 language:
-
-amh
+- am
+license: mit
 tags:
-amharic
-word-embeddings
-skip-gram
-natural-language-processing
-from-scratch
-Amharic Skip-gram Word Embeddings from Scratch
-
-A small Amharic word-embedding model trained using a manually implemented Skip-gram neural network in Python.
-
-Overview
-
-This project implements a Skip-gram model from scratch to learn word representations from a small Amharic text corpus. The goal is to understand and implement the underlying mathematics, including the forward pass, softmax, cross-entropy loss, analytical gradients, and stochastic gradient descent.
-
-The implementation uses Python lists, loops, math, and random. It does not rely on pretrained embeddings, NumPy, automatic differentiation, or neural-network libraries.
-
-Model details
-Property	Value
-Language	Amharic
-Architecture	Skip-gram with softmax
-Vocabulary size	271
-Training pairs	1,976
-Embedding dimension	10
-Context window	2
-Learning rate	0.05
-Maximum epochs	50
-Random seed	42
-Best checkpoint	Epoch 49
-
-The model learns an input embedding matrix E and output weight matrix U. For a center word, it predicts surrounding context words using a softmax distribution over the vocabulary.
-
-Training and evaluation results
-Metric	Result
-Initial mean training loss	5.602120
-Best mean training loss	2.798836
-Best epoch	49
-Epochs executed	50
-Reported maximum absolute gradient-check error	0.00000000
-
-The best mean training loss was approximately 50% lower than the initial mean training loss. The best checkpoint was restored after epoch 50 produced a slightly higher loss.
-
-These are training and implementation-validation results. They are not held-out test metrics, and they do not establish generalization to unseen Amharic text.
-
-Correctness checks
-
-The implementation passed the following checks:
-
-Context-window pair generation.
-Sentence-boundary protection during pair generation.
-Forward scores, softmax probabilities, and cross-entropy loss against a known numerical example.
-Analytical gradient values against the assignment's expected values.
-One stochastic-gradient-descent update, including verification that unrelated input-embedding rows remain unchanged.
-Numerical stability with an extreme score.
-Softmax and loss invariance when a constant is added to every score.
-Numerical gradient checking using finite differences with epsilon 1e-5.
-Cosine similarity handling for zero-norm vectors.
-Model save/reload consistency for vocabulary, parameters, probabilities, and nearest-neighbor order.
-Example use
-
-The repository includes a custom Python implementation rather than a Transformers-compatible model. Load the saved JSON with the accompanying model.py implementation:
-
-from model import SkipGramModel, inverse_vocabulary
-
-model, word_to_id = SkipGramModel.load_json(
-    "amharic_skipgram.json"
-)
-id_to_word = inverse_vocabulary(word_to_id)
-
-word = "ቴክኖሎጂ"
-
-if word in word_to_id:
-    word_id = word_to_id[word]
-    neighbors = model.nearest_neighbors(word_id, top_k=3)
-
-    for neighbor_id, similarity in neighbors:
-        print(id_to_word[neighbor_id], round(similarity, 6))
-else:
-    print("Unknown word:", word)
-
-The similarity values are cosine similarities between learned input embeddings.
-
-Intended uses
-Educational demonstrations of word embeddings and Skip-gram.
-Learning about softmax, cross-entropy, gradient calculations, and SGD.
-Small experiments with Amharic word-context relationships.
-A starting point for further experimentation with Ethiopian-language NLP.
-Limitations
-The corpus is small, and the vocabulary contains only 271 unique words.
-The embedding dimension is only 10.
-Nearest neighbors can be noisy and may reflect local context or grammatical patterns rather than semantic similarity.
-The model has not been evaluated on a held-out benchmark or downstream NLP task.
-This is an educational prototype, not a production-ready or general-purpose Amharic language model.
-Words not present in the vocabulary cannot be queried.
-Dataset and reproducibility
-
-The model was trained on a small Amharic corpus organized around multiple everyday topics. The vocabulary, preprocessing, pair-generation logic, model implementation, and training configuration are documented in the project source code.
-
-Training configuration: window size 2, embedding dimension 10, learning rate 0.05, maximum epochs 50, and random seed 42.
-
-Future work
-Expand the corpus with diverse, properly licensed Amharic text.
-Evaluate on held-out text and compare multiple context-window sizes and embedding dimensions.
-Compare nearest neighbors with judgments from fluent Amharic speakers.
-Add a reproducible training-loss plot and configuration comparison.
-Explore subword handling and improved Amharic text normalization.
-Acknowledgment
-
-Built as an educational assignment to understand and implement Skip-gram word embeddings from first principles.
+- amharic
+- word-embeddings
+- skip-gram
+- natural-language-processing
+- from-scratch
+- custom-implementation
+metrics:
+- loss
+library_name: custom
+---
+
+# Amharic Skip-gram Word Embeddings from Scratch
+
+A lightweight, pure-Python implementation of a Skip-gram word embedding model trained on a small Amharic corpus. Built entirely from first principles without external numerical or machine learning libraries (no NumPy, PyTorch, or TensorFlow).
+
+## Overview
+
+This project implements a Skip-gram neural network to learn vector representations (word embeddings) for Amharic words. The primary goal is educational: to understand and verify the core mathematical foundations of word embeddings, including:
+- Forward propagation and softmax distribution calculation
+- Cross-entropy loss formulation
+- Analytical gradient derivation for input and output weight matrices
+- Stochastic Gradient Descent (SGD) optimization from scratch
+
+The implementation relies strictly on Python standard library modules (`math`, `random`, lists, and loops).
+
+---
+
+## Model Architecture & Training Parameters
+
+| Hyperparameter | Value |
+| :--- | :--- |
+| **Language** | Amharic (`am`) |
+| **Architecture** | Skip-gram with Softmax |
+| **Vocabulary Size** | 271 unique tokens |
+| **Training Pairs** | 1,976 |
+| **Embedding Dimension** | 10 |
+| **Context Window Size** | 2 |
+| **Learning Rate** | 0.05 |
+| **Maximum Epochs** | 50 |
+| **Random Seed** | 42 |
+| **Best Checkpoint** | Epoch 49 |
+
+The model optimizes two matrices: an input embedding matrix $E$ and an output context weight matrix $U$. For a given center word, it predicts surrounding context words via a softmax probability distribution over the vocabulary.
+
+---
+
+## Training & Validation Results
+
+| Metric | Result |
+| :--- | :--- |
+| **Initial Mean Training Loss** | 5.602120 |
+| **Best Mean Training Loss** | 2.798836 |
+| **Best Epoch** | 49 |
+| **Epochs Executed** | 50 |
+| **Max Absolute Gradient-Check Error** | 0.00000000 |
+
+> **Note:** The best mean training loss achieved a ~50% reduction from the initial loss. Epoch 49 was selected via early checkpointing as epoch 50 yielded a slightly higher loss value. These metrics validate implementation correctness on the training split and do not reflect generalization performance on unseen text.
+
+---
+
+## Implementation & Correctness Checks
+
+The model passed a rigorous suite of mathematical and algorithmic unit tests:
+
+- [x] **Data Processing:** Correct context-window pair generation with strict sentence-boundary protection.
+- [x] **Forward & Loss Verification:** Forward scores, softmax probabilities, and cross-entropy loss validated against manual reference calculations.
+- [x] **Analytical Gradients:** Exact match with expected symbolic gradients.
+- [x] **SGD Updates:** Verified parameter updates, ensuring non-context input embedding rows remain unmodified during sparse updates.
+- [x] **Numerical Stability:** Verified stability under extreme score magnitudes and shift invariance ($\text{softmax}(z) = \text{softmax}(z + c)$).
+- [x] **Gradient Checking:** Finite-difference numerical gradient verification ($\epsilon = 10^{-5}$).
+- [x] **Edge Cases:** Safe cosine similarity evaluation for zero-norm vectors.
+- [x] **Serialization:** Full JSON round-trip consistency for vocabulary mapping, weight arrays, and nearest-neighbor outputs.
+
+---
+
+## Usage
+
+This model uses a custom Python execution format rather than Hugging Face `transformers`. Download `amharic_skipgram.json` alongside your `model.py` script to run inference.
+
+### Python Code Example
+
+```python
+from model import SkipGramModel