Spaces:

dkstr
/

drain4-spatiotemporal

Configuration error

App Files Files Community

drain4-spatiotemporal

Commit History

Sep 11

10:35
full-corpus v2 eval results

dkstr

Verified signature
84b3ae1
10:22
eval_v2: mining axis (drain4+param-split vs drain3 per source) + temporal axis (learn 0-2, calibrate on clean day 3, detect 7-13, family-aware sustain gates, counts-only baseline)

dkstr

Verified signature
cddcc61
10:22
temporal v2: parent-keyed families, silence split from counts (presence-scored), sustained-mean spike signal, optional seasonal time-of-day baseline, episode grouping with hysteresis, O(1) rolling correlation

dkstr

Verified signature
aa7a5f4
10:21
calibrated parser: param-split with masked sub-template rendering (template identity = parent), timestamp-shape exclusion, recursive sub-routing, sub-aware persistence

dkstr

Verified signature
a60b12a
09:35
temporal v2: parent-keyed counts/novelty/corr (sub rollup), episode grouping with hysteresis, O(1) rolling co-occurrence, per-family thresholds, MAD=0 spike fix, lazy presence

dkstr

Verified signature
383fb8a
09:35
parser v2: adaptive recursive param-split (cardinality-fitted freeze keys, timestamp-shape exclusion, line-unique ratio rule), sub-aware persistence, sub template IDs anchored to stable parent

dkstr

Verified signature
3281f57
09:11
synthgen v2: realistic sshd background (failed-pw share 18.6%->1.5%) + 6-IP brute-force pool

dkstr

Verified signature
b55b2d1

Sep 10

23:02
Upload results/synth_eval_results.json with huggingface_hub

dkstr

Verified signature
807e937
18:45
eval_synth: absolute-time window indices (relative split made engine never train + labels never intersect); drain3 add_log_message returns dict

dkstr

Verified signature
8217e6f
18:44
temporal: cap per-template entropy history at 1000 samples (unbounded insort was O(N^2) at 100M scale)

dkstr

Verified signature
ed9a47c
18:44
synthgen: source-major task ordering so web shards run in parallel (14-day wall ~15 min)

dkstr

Verified signature
049dd44
18:30
synthgen.py: validated generator (268 templates, 9 sources, 9 incidents, parquet shards) — smoke-tested at scale 0.005

dkstr

Verified signature
be3316d
17:04
synthgen.py: complete clean rewrite — registry, 9 sources, incidents, parquet shards, meta.json

dkstr

Verified signature
e472930
16:52
Complete synthgen implementation (replaces truncated draft)

dkstr

Verified signature
b5b963d
16:48
Synthetic multi-source log generator: ~500 templates, Zipf skew, diurnal+bursty arrivals, 9 injected incident types, parquet shards

dkstr

Verified signature
1d93b2e
16:20
benchmark results: eval_temporal_results.json

dkstr

Verified signature
87e6a42
16:20
benchmark results: benchmark_results.json

dkstr

Verified signature
84cbbc6
16:15
Fix: eng.update=False before detect; m3.drain.clusters; rarity signal added to BGL axis

dkstr

Verified signature
cffd96a
16:15
Perf + correctness: incremental novelty bookkeeping (first/last nonzero, gaps), trailing-window stats cap

dkstr

Verified signature
46c818d
16:10
Fix parquet path: default/partial-test (verified 74,273 rows)

dkstr

Verified signature
eeabf2e
16:08
Zookeeper source -> bolu61/loghub_2 test parquet (full 74k-line Zookeeper); two-window silence; raw content passed; deterministic injection placement

dkstr

Verified signature
08ba21a
16:00
Fix infinite loop: advance current_window after closing (sub-agent found hang in _close_window loop)

dkstr

Verified signature
4965a52
15:49
Rewrite: robust_stats public, first-sighting fix, per-instance entropy history, bounded corr scan

dkstr

Verified signature
c290ddb
15:49
README

dkstr

Verified signature
f53b040
15:49
package init

dkstr

Verified signature
ceccd30
15:49
Clean rewrite: single inline-injection replay + drain3 z-score baseline + BGL novelty AUC

dkstr

Verified signature
1d38521
15:46
Spatiotemporal anomaly engine: 4 signal families + observability/security fusion

dkstr

Verified signature
240fa22
15:46
Fix tree_search wildcard fallback; simplify LRU dict

dkstr

Verified signature
a62b46a
15:46
Temporal anomaly benchmark: injected anomalies on Zookeeper stream + BGL novelty AUC

dkstr

Verified signature
5045f24
15:46
Template-parsing benchmark: drain4 vs pip drain3 on LogHub 2k ground truth

dkstr

Verified signature
003c5a8
15:44
Core Drain reproduction: fixed-depth prefix tree, position/LCS similarity, stable template IDs, JSON persistence

dkstr

Verified signature
ffff982
15:43
initial commit

dkstr

Verified signature
0cd2d69

Commit History

Sep 11

full-corpus v2 eval results

eval_v2: mining axis (drain4+param-split vs drain3 per source) + temporal axis (learn 0-2, calibrate on clean day 3, detect 7-13, family-aware sustain gates, counts-only baseline)

temporal v2: parent-keyed families, silence split from counts (presence-scored), sustained-mean spike signal, optional seasonal time-of-day baseline, episode grouping with hysteresis, O(1) rolling correlation

calibrated parser: param-split with masked sub-template rendering (template identity = parent), timestamp-shape exclusion, recursive sub-routing, sub-aware persistence

temporal v2: parent-keyed counts/novelty/corr (sub rollup), episode grouping with hysteresis, O(1) rolling co-occurrence, per-family thresholds, MAD=0 spike fix, lazy presence

parser v2: adaptive recursive param-split (cardinality-fitted freeze keys, timestamp-shape exclusion, line-unique ratio rule), sub-aware persistence, sub template IDs anchored to stable parent

synthgen v2: realistic sshd background (failed-pw share 18.6%->1.5%) + 6-IP brute-force pool

Sep 10

Upload results/synth_eval_results.json with huggingface_hub

eval_synth: absolute-time window indices (relative split made engine never train + labels never intersect); drain3 add_log_message returns dict

temporal: cap per-template entropy history at 1000 samples (unbounded insort was O(N^2) at 100M scale)

synthgen: source-major task ordering so web shards run in parallel (14-day wall ~15 min)

synthgen.py: validated generator (268 templates, 9 sources, 9 incidents, parquet shards) — smoke-tested at scale 0.005

synthgen.py: complete clean rewrite — registry, 9 sources, incidents, parquet shards, meta.json

Complete synthgen implementation (replaces truncated draft)

Synthetic multi-source log generator: ~500 templates, Zipf skew, diurnal+bursty arrivals, 9 injected incident types, parquet shards

benchmark results: eval_temporal_results.json

benchmark results: benchmark_results.json

Fix: eng.update=False before detect; m3.drain.clusters; rarity signal added to BGL axis

Perf + correctness: incremental novelty bookkeeping (first/last nonzero, gaps), trailing-window stats cap

Fix parquet path: default/partial-test (verified 74,273 rows)

Zookeeper source -> bolu61/loghub_2 test parquet (full 74k-line Zookeeper); two-window silence; raw content passed; deterministic injection placement

Fix infinite loop: advance current_window after closing (sub-agent found hang in _close_window loop)

Rewrite: robust_stats public, first-sighting fix, per-instance entropy history, bounded corr scan

README

package init

Clean rewrite: single inline-injection replay + drain3 z-score baseline + BGL novelty AUC

Spatiotemporal anomaly engine: 4 signal families + observability/security fusion

Fix tree_search wildcard fallback; simplify LRU dict

Temporal anomaly benchmark: injected anomalies on Zookeeper stream + BGL novelty AUC

Template-parsing benchmark: drain4 vs pip drain3 on LogHub 2k ground truth

Core Drain reproduction: fixed-depth prefix tree, position/LCS similarity, stable template IDs, JSON persistence

initial commit