File size: 1,768 Bytes
dfb775d | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 | """mindX Efficiency Index (MEI) — composite model-evaluation metric.
Implements the v0.1 specification at
`/home/hacker/mindX/docs/operations/mindX Efficiency Index_ … .md`.
The MEI is a single scalar in [0, 1] derived from five log-compressed
sub-indices — quality (Q), decode throughput (Dt), prefill throughput
(Pp), memory footprint (M), and energy per useful token (E) — combined
via weighted geometric mean. It is the in-house metric mindXtrain uses
to gate checkpoint promotion to AgenticPlace.
Layout (mirrors the spec's §7 three-layer instrumentation stack):
- `record.py` — Pydantic schemas for the canonical measurement record.
- `tokenizer.py` — canonical tokenizer wrapper (spec Rule 3.1).
- `throughput.py` — engine harness wrappers (llama.cpp, Ollama, vLLM, HF).
- `energy.py` — power sampling (NVML, RAPL, powermetrics, ROCm-SMI).
- `orchestrator.py` — measurement protocol: warmup → battery → sweep → CI.
- `score.py` — pure functions computing MEI from a record.
- `xei.py` — training-side companion (MFU, optimization health).
- `promotion.py` — three-gate AgenticPlace promotion logic.
- `history.py` — append-only JSONL of historical scores.
Anchor calibration (§5.2-§5.5) is frozen at module load for v0.1; the
ANCHORS object is the authoritative source for floors and ceilings.
"""
from mindxtrain.eval.mei.record import (
ConcurrencyPoint,
ContextTierMeasurement,
HardwareIdent,
InferenceEngineIdent,
LatencyPercentiles,
MEIRecord,
QuantizationTuple,
TokenSeries,
)
__all__ = [
"ConcurrencyPoint",
"ContextTierMeasurement",
"HardwareIdent",
"InferenceEngineIdent",
"LatencyPercentiles",
"MEIRecord",
"QuantizationTuple",
"TokenSeries",
]
|