Engineering Online Experimentation and ML Evaluations
Regulärer Preis:
62,99 €
Sofort verfügbar
Engineering Online Experimentation and ML Evaluations, Apress
Architecture, Statistics and Machine Learning for Production-Scale Systems
Von Ming Lei, im heise shop in digitaler Fassung erhältlich
Produktinformationen "Engineering Online Experimentation and ML Evaluations"
Online experimentation is now essential for modern software and machine learning
teams. This book provides an engineer-first, end-to-end guide to building and
operating production-ready experimentation platforms.
The book begins with Part I establishing the core foundations of credible
experimentation, including hypothesis testing, power analysis, sample sizing,
metric design, and common pitfalls such as peeking, multiple testing, and
novelty or learning effects. Part II focuses on platform
engineering—traffic and identity management, mutual exclusion, event and
logging design, ETL/ELT pipelines, building a stats engine with SciPy and
statsmodels, SRM detection, integrating deployments with feature flags and
canaries, and setting up guardrail and health monitoring. Part III presents
advanced designs that improve speed and sensitivity: sequential testing with
alpha spending, bootstrap intervals for ratios and quantiles, A/B/n testing with
ANOVA, interleaving for ranking systems, switchback and geo experiments, and
multi-armed bandits. Part IV connects experimentation to ML workflows, covering
offline, shadow, canary, and A/B evaluation pipelines; Bayesian optimization for
adaptive experimentation; counterfactual and IPS methods for learning from logs;
and safe retraining supported by strong governance.
What you will learn:
- Design trustworthy experiments with proper metrics, guardrails, α/power/MDE settings, and safeguards against peeking and multiple-testing errors
- Build a production-ready experimentation stack with assignment, identity/diversion, logging, ETL/ELT, a stats engine, and SRM checks
- Run advanced designs at scale, including sequential tests, bootstrap CIs, interleaving, switchback/geo experiments, and multi-armed bandits
- Evaluate ML systems from offline to online, leverage experiment logs for learning, and enable safe retraining with governance
Artikel-Details
- Anbieter:
- Apress
- Autor:
- Ming Lei
- Artikelnummer:
- 9798868827211
- Veröffentlicht:
- 29.07.26
Barrierefreiheit
This PDF has been created in accordance with the PDF/UA-1 standard to enhance accessibility, including screen reader support, described non-text content (images, graphs), bookmarks for easy navigation
- entspricht den Vorgaben der PDF / UA 1 (05)
- keine Vorlesefunktionen des Lesesystems deaktiviert (bis auf) (10)
- navigierbares Inhaltsverzeichnis (11)
- logische Lesereihenfolge eingehalten (13)
- kurze Alternativtexte (z.B für Abbildungen) vorhanden (14)
- Inhalt auch ohne Farbwahrnehmung verständlich dargestellt (25)
- hoher Kontrast zwischen Text und Hintergrund (26)
- Navigation über vor-/zurück-Elemente (29)
- alle zum Verständnis notwendigen Inhalte über Screenreader zugänglich (52)
- Kontakt zum Herausgeber für weitere Informationen zur Barrierefreiheit (99)