EISV-Lumen Student -- Distilled RandomForest for Edge Deployment
A lightweight RandomForest ensemble distilled from the EISV-Lumen Teacher (LoRA on Qwen3-4B). Achieves 0.986 coherence on the EISV expression-generation task while fitting in ~13 MB of JSON with zero external dependencies -- only Python stdlib required. Designed to run on a Raspberry Pi 4 (Lumen's physical host).
Model Details
| Field | Value |
|---|---|
| Method | Knowledge distillation (teacher-labeled soft targets) |
| Architecture | 3 independent RandomForest classifiers (sklearn) |
| Input features | 12 numeric (EISV means, deltas, accelerations) + 9 shape one-hot |
| Training data | 4,320 teacher-labeled examples (9 shapes x 480 each) |
| Test data | 1,080 held-out examples |
| Format | Zero-dependency JSON (~13 MB) |
| Target hardware | Raspberry Pi 4 (1.5 GHz ARM, 4 GB RAM) |
How It Works
The student decomposes EISV expression generation into three chained classification problems, each solved by an independent RandomForest:
- Pattern classifier -- predicts one of 5 expression patterns:
SINGLE,PAIR,REPETITION,QUESTION,TRIPLE - Token-1 classifier -- predicts the primary EISV token from 15
classes (e.g.,
~stillness~,~warmth~,~emergence~) - Token-2 classifier -- predicts the secondary token from 15 + none, conditioned on the Token-1 prediction (token1 index appended as extra feature)
The pattern determines how tokens are assembled into the final expression
string (e.g., PAIR yields two distinct tokens, REPETITION repeats
token-1 twice).
Results
| Metric | Student (RF) | Teacher (Qwen3-4B LoRA) | Random Baseline |
|---|---|---|---|
| Coherence | 0.986 | 0.952 | 0.495 |
| Token-1 agreement | 0.688 | -- | -- |
| Pattern agreement | 0.652 | -- | -- |
| Full agreement (all 3 match) | 0.403 | -- | -- |
Why does the student exceed the teacher? The RandomForest decision boundaries naturally cluster predictions toward high-affinity tokens for each trajectory shape. While the student disagrees with the teacher on exact token choices ~30% of the time, the tokens it picks are still coherent -- they belong to the same affinity region of EISV space. The coherence metric rewards any valid expression, not exact match.
The two coherence figures also come from different test sets: the student's from 1,080 held-out teacher-labelled synthetic windows, the teacher's from 500 real Lumen windows. Read the comparison as indicative, not as a like-for-like result.
Zero-Dependency Usage (recommended for edge)
The exported/ directory contains JSON-serialized forests and a standalone
inference module. No pip packages required.
from student_inference import StudentInference
student = StudentInference("path/to/exported/")
result = student.predict("settled_presence", {
"mean_E": 0.7, "mean_I": 0.6, "mean_S": 0.2, "mean_V": 0.05,
"dE": 0.0, "dI": 0.0, "dS": 0.0, "dV": 0.0,
"d2E": 0.0, "d2I": 0.0, "d2S": 0.0, "d2V": 0.0,
})
# result = {"pattern": "SINGLE", "eisv_tokens": ["~stillness~"],
# "token_1": "~stillness~", "token_2": "none"}
Only json and os from the standard library are used. The inference
module walks each decision tree node-by-node and averages class
probabilities across all trees -- identical to sklearn's predict logic.
File Structure
eisv-lumen-student/
|-- README.md # This file
|-- training_metrics.json # Cross-validation metrics
|-- eval_results.json # Full evaluation results
|-- exported/ # Zero-dependency JSON format
|-- pattern_forest.json # Decision trees as JSON (3.0 MB)
|-- token1_forest.json # Decision trees as JSON (4.5 MB)
|-- token2_forest.json # Decision trees as JSON (5.1 MB)
|-- scaler.json # Scaler parameters (511 B)
|-- mappings.json # Label mappings (1.1 KB)
|-- student_inference.py # Standalone inference (4.9 KB)
Training Details
- Distillation source: Teacher (Qwen3-4B LoRA v6, 0.952 coherence on real Lumen trajectories)
- Data generation: 4,320 synthetic EISV trajectories labeled by teacher inference (480 per shape x 9 shapes), plus 1,080 held-out test examples
- Forest hyperparameters:
n_estimators=100,max_depth=None,random_state=42(sklearn defaults) - Feature engineering: 12 numeric features (4 EISV means + 4 first
derivatives + 4 second derivatives) standardized via
StandardScaler, plus 9-dimensional one-hot encoding of trajectory shape
Related
- Teacher model: hikewa/eisv-lumen-teacher -- Qwen3-4B LoRA that generated the training labels
- Dataset: hikewa/unitares-eisv-trajectories -- EISV trajectory data from Lumen
- Explorer Space: hikewa/eisv-lumen-explorer -- interactive demo
Citation
@misc{eisv-lumen-student-2026,
title = {EISV-Lumen Student: Distilled RandomForest for Edge Deployment},
author = {Wang, Kenny},
year = {2026},
url = {https://huggingface.co/hikewa/eisv-lumen-student},
note = {Knowledge-distilled RandomForest ensemble for EISV expression
generation on Raspberry Pi}
}
- Downloads last month
- 24