SRE Incident Intelligence Engine — Pareto 80/20, MAD Z-Score & Dash
Clean-room analytical engine built in Python (Pandas, NumPy, Plotly, Dash, Gunicorn) to transform raw Tier-2 / Tier-3 (L3) incident queues into statistically rigorous engineering prioritization (Pareto 80/20, contractual SLA compliance matrices, skewed MTTR percentiles, and robust MAD anomaly detection) — powered 100% by deterministic synthetic telemetry (Zero NDA / Zero PII).
- Public Repository & CI (
Python 3.11 / 3.12+Docker): github.com/evslopes/sre-incident-analytics-dash - Architecture Decision Records (
ADRs 0001–0003): github.com/evslopes/sre-incident-analytics-dash/tree/main/docs/adr
Dual-Surface Architecture: Live L3 Investigation & Executive Storytelling
Mission-critical sustainment teams need both interactive multidimensional drill-down and zero-server Data Storytelling artifacts for weekly SLA and Six Sigma governance reviews. Both surfaces share a single pure analytical core:
Synthetic Generator (Seed=42) / CSV Adapter
Schema & Null Audit
Leakage-Free Backlog
SRE KPI Engine
Pareto, SLA Matrix, P50/P90/P95 & MAD
Reactive Plotly Dash Console
Gunicorn :8050
Standalone Zero-Server HTML Report Exporter
Statistical Rigor & Operational Data Hygiene
| Operational Analytics Challenge | Classic Implementation Pitfall | Engineering Approach Implemented |
|---|---|---|
Right-Skewed Repair Times (MTTR) | Reporting arithmetic means distorted by single long-tail incidents | Systematic P50, P90, and P95 percentiles paired with log-scale severity distributions (SEV-1..SEV-4) |
Post-Deployment (GMUD) Spike Detection | Gaussian Z-scores ($\mu \pm 3\sigma$) suffer statistical masking during extreme bursts | Modified Z-Score via MAD (Median Absolute Deviation, threshold $> 3.5$) with automatic IQR / 1.349 fallback |
Active Unresolved Backlog (NULLs) | Dropping rows without resolved_at (hiding in-flight SLA breaches) or leaking future timestamps | Explicit DataQualityAudit preserves active backlog, evaluates in-flight SLA elapsed time (reference_now - opened_at), labels "Pending L3 RCA", and imputes missing MTTA via per-severity median with ack_imputed flag |
| Six Sigma / DMAIC Toil Elimination | Treating incidents individually rather than clustering systemic failure modes | Composite service :: root_cause_category fingerprinting isolating the Vital Few driving 80% of toil and synthesizing P0 / P1 / P2 actions |
Modular package featuring a unified CLI (sre-analytics), Pytest suite, non-root Gunicorn Dockerfile, and statistical ADRs.