yadelew commited on
Commit
6fbbd1d
·
verified ·
1 Parent(s): 39ec07b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -24
README.md CHANGED
@@ -61,28 +61,6 @@ The model optimizes two matrices: an input embedding matrix $E$ and an output co
61
 
62
  > **Note:** The best mean training loss achieved a ~50% reduction from the initial loss. Epoch 49 was selected via early checkpointing as epoch 50 yielded a slightly higher loss value. These metrics validate implementation correctness on the training split and do not reflect generalization performance on unseen text.
63
 
64
- ---
65
-
66
- ## Implementation & Correctness Checks
67
-
68
- The model passed a rigorous suite of mathematical and algorithmic unit tests:
69
-
70
- - [x] **Data Processing:** Correct context-window pair generation with strict sentence-boundary protection.
71
- - [x] **Forward & Loss Verification:** Forward scores, softmax probabilities, and cross-entropy loss validated against manual reference calculations.
72
- - [x] **Analytical Gradients:** Exact match with expected symbolic gradients.
73
- - [x] **SGD Updates:** Verified parameter updates, ensuring non-context input embedding rows remain unmodified during sparse updates.
74
- - [x] **Numerical Stability:** Verified stability under extreme score magnitudes and shift invariance ($\text{softmax}(z) = \text{softmax}(z + c)$).
75
- - [x] **Gradient Checking:** Finite-difference numerical gradient verification ($\epsilon = 10^{-5}$).
76
- - [x] **Edge Cases:** Safe cosine similarity evaluation for zero-norm vectors.
77
- - [x] **Serialization:** Full JSON round-trip consistency for vocabulary mapping, weight arrays, and nearest-neighbor outputs.
78
-
79
- ---
80
-
81
- ## Usage
82
-
83
- This model uses a custom Python execution format rather than Hugging Face `transformers`. Download `amharic_skipgram.json` alongside your `model.py` script to run inference.
84
-
85
- ### Python Code Example
86
 
87
- ```python
88
- from model import SkipGramModel
 
61
 
62
  > **Note:** The best mean training loss achieved a ~50% reduction from the initial loss. Epoch 49 was selected via early checkpointing as epoch 50 yielded a slightly higher loss value. These metrics validate implementation correctness on the training split and do not reflect generalization performance on unseen text.
63
 
64
+ ### Training Loss Curve
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
65
 
66
+ The following plot illustrates the trajectory of the mean training loss recorded before training and after each epoch. The best model checkpoint was saved at epoch 49: