Update README.md
Browse files
README.md
CHANGED
|
@@ -61,28 +61,6 @@ The model optimizes two matrices: an input embedding matrix $E$ and an output co
|
|
| 61 |
|
| 62 |
> **Note:** The best mean training loss achieved a ~50% reduction from the initial loss. Epoch 49 was selected via early checkpointing as epoch 50 yielded a slightly higher loss value. These metrics validate implementation correctness on the training split and do not reflect generalization performance on unseen text.
|
| 63 |
|
| 64 |
-
|
| 65 |
-
|
| 66 |
-
## Implementation & Correctness Checks
|
| 67 |
-
|
| 68 |
-
The model passed a rigorous suite of mathematical and algorithmic unit tests:
|
| 69 |
-
|
| 70 |
-
- [x] **Data Processing:** Correct context-window pair generation with strict sentence-boundary protection.
|
| 71 |
-
- [x] **Forward & Loss Verification:** Forward scores, softmax probabilities, and cross-entropy loss validated against manual reference calculations.
|
| 72 |
-
- [x] **Analytical Gradients:** Exact match with expected symbolic gradients.
|
| 73 |
-
- [x] **SGD Updates:** Verified parameter updates, ensuring non-context input embedding rows remain unmodified during sparse updates.
|
| 74 |
-
- [x] **Numerical Stability:** Verified stability under extreme score magnitudes and shift invariance ($\text{softmax}(z) = \text{softmax}(z + c)$).
|
| 75 |
-
- [x] **Gradient Checking:** Finite-difference numerical gradient verification ($\epsilon = 10^{-5}$).
|
| 76 |
-
- [x] **Edge Cases:** Safe cosine similarity evaluation for zero-norm vectors.
|
| 77 |
-
- [x] **Serialization:** Full JSON round-trip consistency for vocabulary mapping, weight arrays, and nearest-neighbor outputs.
|
| 78 |
-
|
| 79 |
-
---
|
| 80 |
-
|
| 81 |
-
## Usage
|
| 82 |
-
|
| 83 |
-
This model uses a custom Python execution format rather than Hugging Face `transformers`. Download `amharic_skipgram.json` alongside your `model.py` script to run inference.
|
| 84 |
-
|
| 85 |
-
### Python Code Example
|
| 86 |
|
| 87 |
-
|
| 88 |
-
from model import SkipGramModel
|
|
|
|
| 61 |
|
| 62 |
> **Note:** The best mean training loss achieved a ~50% reduction from the initial loss. Epoch 49 was selected via early checkpointing as epoch 50 yielded a slightly higher loss value. These metrics validate implementation correctness on the training split and do not reflect generalization performance on unseen text.
|
| 63 |
|
| 64 |
+
### Training Loss Curve
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 65 |
|
| 66 |
+
The following plot illustrates the trajectory of the mean training loss recorded before training and after each epoch. The best model checkpoint was saved at epoch 49:
|
|
|