Machine Learning Systems: Volume I v0.7.0 and Volume II v0.2.0

Machine Learning Systems: Volume I v0.7.0 and Volume II v0.2.0

This release ships a comprehensive editorial and production pass across both volumes, refreshing the HTML, PDF, and EPUB builds. It also answers the question we hear most often, so this note opens with a clear guide to what each volume covers and which one to read.

---

📚 Which volume should I read? Volume I vs Volume II

Short answer: Volume I teaches you how a machine learning system works and how to build one. Volume II teaches you what changes when machine learning runs at scale, across many machines, many users, and long-lived production fleets. Volume I is about the single system. Volume II is about the fleet.

The two volumes are separate, self-contained reading paths. Volume I assumes no prior systems background and is the place to start. Volume II assumes you are comfortable with the foundations from Volume I and goes deep on large-scale and production concerns.

	Volume I	Volume II
Title	Introduction to Machine Learning Systems	Machine Learning Systems at Scale
Version	v0.7.0	v0.2.0
Core question	How do you build, train, optimize, and deploy a machine learning system?	How do you run machine learning across thousands of machines, reliably, efficiently, and responsibly?
Unit of thinking	One model, one system, end to end	A fleet: datacenters, clusters, and distributed services
Prerequisites	None beyond basic programming and curiosity	The foundations covered in Volume I
Best for	Students and newcomers building a complete mental model of ML systems	Readers ready for distributed training, large-scale serving, and production operations

Volume I, Introduction to Machine Learning Systems (v0.7.0)

The foundational volume. It builds the complete mental model of how a machine learning system is engineered, from raw data to a deployed, maintained model, working from first principles rather than from any single framework. It is organized into four parts:

• Foundations. What a machine learning system actually is, the workflow that produces one from end to end, and the data engineering that feeds it.
• Build. Neural computation, neural network architectures, the frameworks that express them, and how training works.
• Optimize. Choosing and curating data, compressing models, accelerating them with hardware, and measuring all of it through benchmarking.
• Deploy. Serving models in production, the operations practices (MLOps) that keep them running, and the responsibilities of the engineer who builds them.

If you are new to the field, or you want one coherent picture of how ML systems fit together, start here. Volume I stands on its own.

Volume II, Machine Learning Systems at Scale (v0.2.0)

The advanced volume. It picks up where Volume I leaves off and tackles the problems that only appear once a single system becomes a fleet: many accelerators, many nodes, many users, and the economics and reliability that come with them. It is organized into four parts:

• Fleet Infrastructure. The physical building blocks of scale: compute infrastructure, the network fabrics that connect it, and the data and storage systems that feed it.
• Distributed Machine Learning. Distributed training, collective communication, fault tolerance, and orchestrating work across a fleet.
• Deployment at Scale. Performance engineering, large-scale inference, edge intelligence, and the operations practices that hold up at fleet scale.
• Responsible AI at Scale. Security and privacy, robustness, sustainability, and responsible AI when systems operate across many users and environments.

Read Volume II once the foundations from Volume I feel comfortable. It is written for readers who want to understand how production and datacenter-scale machine learning really works.

Coming from the earlier single-volume PDF?

The earlier single PDF has become these two volumes. The material has been substantially revised and expanded, not simply split, so chapters are deeper and several topics are new. The recommended path is to switch to the volumes: begin with Volume I to confirm the foundations, then continue into Volume II for the scale material that the original edition did not cover in this depth.

The earlier single-volume edition is not going away. We are keeping it archived and available, so if you have notes, citations, lecture materials, or links built on it, that work is not lost. New readers should start with the two volumes above.

---

✨ What changed from the previous release to this one

The previous public release was Volume I v0.6.2 and Volume II v0.1.2. Since then, both volumes have gone through a full, chapter-by-chapter review in canonical reading order. This was the largest editorial and production pass the book has had: every chapter of both volumes was read, audited, and corrected across the dimensions below. The detailed, itemized notes follow.

Highlights

Three areas saw especially large improvements this release:

• Narrative war stories (the book's lore). The real-world story callouts threaded through the chapters, 35 across the two volumes, such as the COMPAS recidivism audit, the PaLM training loss spikes, the Strava heatmap privacy leak, and Apple Card's missing explanations, were substantially reworked. Each one was rewritten to land squarely on the systems lesson it teaches, to follow a consistent narrative beat from problem to resolution, and to tie into its chapter's central thesis. These are the stories that make the engineering memorable, and they got a lot of attention.
• Callouts. The book's roughly one thousand callout boxes (definitions, examples, takeaways, perspectives, quizzes, war stories, and more) were reviewed in their rendered context for placement, flow, and labeling, so each box is bridged into the surrounding prose instead of interrupting it.
• Computed LEGO blocks. Hundreds of executable LEGO code blocks now compute the numbers, tables, and figures shown in the text directly. The values a reader sees are produced by running code rather than typed in by hand, so they cannot drift out of sync with the text. This was the single largest correctness pass in the release.

Content and pedagogy

• Full audit of both volumes. Every chapter was reviewed for prose flow, progressive disclosure (introducing each idea only after the vocabulary it depends on), emphasis and bolding, chapter openings, and conclusions, so each chapter reads cleanly from start to finish and reinforces its core principles.
• Object integration. Figures, tables, listings, captions, callouts, and code outputs are now introduced and interpreted by the surrounding narrative rather than dropped in. Tables and figures carry captions and cross-references so they can be cited precisely from the text.
• Prose clarity passes. A sentence-level read-through of every chapter, a reader-simulation pass that follows a student reading in order to find places where an idea arrives before its setup, and a fold-in pass that absorbed orphaned one-sentence paragraphs into the text around them.
• Boundary and scope corrections. Sharpened where related topics live across chapters (for example, keeping physical compute building blocks distinct from operations, cost, and management) so each chapter owns its material without overlap.

Notation, references, and consistency

• Notation and emphasis standardized across both volumes, with consistent term usage tracked through a shared registry so the same concept is named the same way throughout.
• Cross-references for tables, figures, listings, and equations checked for correctness and consistent casing, with orphaned labels given natural prose references.
• References, citations, and bibliography tidied, with footnotes and external links verified.

Production and layout

• PDF layout fixed for release: margin-note collisions resolved, object overflow off the page cleared, table and callout spacing tightened, figures resized to fit, and cross-reference numbering corrected, checked by rendering and inspecting the actual pages.
• HTML, PDF, and EPUB builds refreshed and verified for both volumes.

Release process

• Exercised the full release audit workflow end to end before publishing.
• Refreshed the companion MLSysIM, slides, instructor, labs, kits, TinyTorch, StaffML, and landing sites as part of the same release pass.

---

🔢 Versions

• Volume I (Introduction to Machine Learning Systems): v0.6.2 to v0.7.0
• Volume II (Machine Learning Systems at Scale): v0.1.2 to v0.2.0

🌐 Access

• Online (HTML): mlsysbook.ai
• PDF: available from the release assets, one file per volume
• EPUB: available from the release assets, one file per volume

🙏 A note from me

I want to say this part personally, because this book has been a labor of love for almost three years.

I split it into two volumes because I finally accepted that this subject is genuinely complicated, and that pretending otherwise does not serve you. Building a machine learning system and running it at scale are two different skills, each one big enough to deserve its own volume. A lot of that conviction came from things most readers never see, like the interviews I do behind the scenes, the conversations with engineers building these systems, and the feedback from industry practitioners and academics who live this work every day. Many of them told me they wanted the split too. It is how we already teach hard subjects. You learn operating systems before distributed systems, and computer architecture before advanced microprocessor architecture.

The piece I am most proud of is the war stories, the real-world narratives woven through the chapters. I spent a lot of time making sure each one lands on the systems l...

TinyTorch v0.1.13: Framework Correctness and Release Pipeline Hardening

TinyTorch v0.1.13

Patch release focused on framework correctness. It lands autograd, convolution, attention, and quantization fixes that close real gradient and numerical stability bugs, plus a substantial hardening of the release and grading pipeline (nbgrader tiers, tito preflight, PDF builds) and a round of community dashboard repairs. Version bumped from 0.1.12 to 0.1.13.

✨ New Features

• AvgPool2d autograd backward (M09 convolutions): AvgPool2d now participates in the autograd graph, distributing the upstream gradient uniformly across each pooling window so convolutional milestones can be trained from input to loss. (PR by @vedant-a-joshi)

🐛 Framework Correctness (autograd and ops)

The core of this release. Each fix addresses a case where the framework silently produced wrong gradients or values:

• MatmulBackward for 1D vector inputs (M06 autograd): corrected gradient computation when one or both operands are 1D. The vector and matrix broadcasting in the backward pass was previously mishandled.
• tracked_mul scalar handling (M06 autograd, #1871): now wraps a raw scalar in a Tensor before passing it to MulBackward instead of leaking a Python float into the graph.
• Conv2dBackward return tuple (M09 convolutions, #1866): the backward function's return arity now matches the length of saved_tensors, fixing gradient unpacking errors.
• Stray super().__init__() removed from Conv2d, MaxPool2d, and AvgPool2d (M09, #1868): these misleading calls implied an inheritance contract that does not exist in the TinyTorch layer model.
• Numerically stable sigmoid (M02 activations): reformulated to avoid exp overflow warnings on large magnitude inputs.
• Quantization of large constants (M15 quantization): constant tensors with |value| > 127 are now recovered correctly instead of being clipped or lost during the quantize round trip.
• MultiHeadAttention mask reshape (M12 attention): preserves seq_q and seq_k dimensions so the attention mask keeps the correct shape across multi-head reshaping.
• GPT causal mask convention (M13 transformers): aligned the causal mask orientation so decoder self-attention masks future positions correctly.
• reshape(-1) divisibility check (M01 tensor, #1872): validates that the known dimensions evenly divide the element count before inferring the missing dimension, replacing a silent miscompute with a clear error.
• Dropout RNG (M03 layers, #1869): Dropout now draws from the module level seeded RNG instead of global np.random, restoring reproducibility under a fixed seed.

🔬 Tests

• Transpose test hardened (M01 tensor, #1873): switched to a non symmetric tensor so the test actually catches incorrect axis swaps (a symmetric tensor masked the bug).
• CLI progress sync tests (community): added coverage for the path that syncs tito progress to the community dashboard. (by @jettythek)
• tito module complete now catches notebook syntax errors and runs integration tests as part of completion, so broken submissions fail loudly.

🔧 Tito CLI and Release / Grading Pipeline

A broad hardening pass so lab artifacts build, grade, and publish consistently:

• nbgrader release tiers: defined and documented hidden nbgrader release tiers, added them to tito, hardened release staging, and restored and aligned the nbgrader release workflows.
• tito dev preflight command registered for validation before a release.
• Reliable progress sync (#1849): tito CLI progress now syncs dependably to the community dashboard.
• nbdev config (#1876): added [tool.nbdev] to pyproject.toml so nbdev reads its configuration directly instead of falling back to defaults. Also pinned nbdev < 3.0.16 for stability.
• PDF builds: locate the TinyTeX tlmgr binary so the lab guide and research paper PDFs build cleanly in CI.
• Dependency floors: aligned the jupyterlab minimum version.
• Repaired release regressions and refreshed the release narrative.

🌐 Community Dashboard and Site

• Globe performance (community): redrew the contributor globe on a canvas for smoother performance and fixed profile setup logic. (by @jettythek)
• Map legend and data fetching improved on the community map. (by @jettythek)
• Profile setup and email login dashboard access repaired (#1887) so logins by email can complete setup and reach the dashboard. (by @farhan523)
• Sidebar textbook link now navigates to mlsysbook.ai.
• 404 dark mode: added body.quarto-dark selectors alongside the @media dark rules so the 404 page respects the manual theme toggle.

📚 Documentation

• Expanded preface with dedicated Prerequisites, Instructor, and Contribute sections, clarifying who the course is for and how to get involved.
• nbgrader release tier documentation and a refreshed release narrative (see above).

📦 Dependencies

• Bumped @types/vscode and @types/node in the VS Code extension.
• Updated the jupytext requirement (#1758).
• Applied bib-tidy automatic formatting.

---

👥 Contributors

TinyTorch is built by its community, and this release is a great example of that. Heartfelt thanks to everyone who showed up, fixing gradients, hardening the pipeline, and making the dashboard work better for the next learner who walks in the door:

• @profvjreddi: release pipeline hardening (nbgrader tiers, tito preflight, PDF builds), preface expansion, and editorial direction
• @Shashank-Tripathi-07: the framework correctness pass (autograd, convolutions, attention, quantization, dropout, reshape, and transpose)
• @jettythek: community dashboard globe and map performance, plus CLI progress sync tests
• @vedant-a-joshi: AvgPool2d autograd backward
• @farhan523: community profile setup and email login dashboard fixes

Every fix here makes TinyTorch a little clearer and more correct for students learning ML systems from the ground up. If you spotted something while working through the modules, we would love your contribution too. Issues and PRs are always welcome. 💚

---

Full Changelog: tinytorch-v0.1.12...tinytorch-v0.1.13

Website: https://mlsysbook.ai/tinytorch/

Lecture Slides (Latest)

ML Systems Lecture Slides

Complete slide decks for the ML Systems curriculum.

Volume I (17 decks): Introduction through Conclusion — single-machine ML systems
Volume II (18 decks): Introduction through Conclusion — distributed ML at scale
TinyML (5 chapters): HarvardX Professional Certificate — edge & embedded ML

Vol I/II decks include speaker notes, active learning exercises, and original SVG diagrams.

Downloads — PDF

• MLSysBook-Slides-All-PDF.zip — All 35 Vol I/II decks
• MLSysBook-Slides-Vol1-PDF.zip — Volume I only
• MLSysBook-Slides-Vol2-PDF.zip — Volume II only

Downloads — PowerPoint

Image-based PPTX (not editable text) — use for presenter mode and annotations.

• MLSysBook-Slides-All-PPTX.zip — All 35 Vol I/II decks
• MLSysBook-Slides-Vol1-PPTX.zip — Volume I only
• MLSysBook-Slides-Vol2-PPTX.zip — Volume II only

Downloads — TinyML (HarvardX edX)

• MLSysBook-TinyML-All.zip — All slides, readings, and supplementary materials
• MLSysBook-TinyML-Slides.zip — 178 slide decks only
• MLSysBook-TinyML-Readings.zip — 127 readings only

Individual Vol I/II chapter files (PDF + PPTX) are also attached below.

Source

• Vol I/II LaTeX source: slides/
• TinyML courseware: tinyMLx/courseware

TinyTorch v0.1.10 — Publication-Grade Lab Guide

TinyTorch v0.1.10

Largest Lab Guide upgrade since the series began, plus substantive framework work: new Tensor API surface (view, masked_fill, ndim, numel, contiguous), no_grad() context manager, Python 3.10+ baseline, 28 security alerts resolved, seven batches of module-audit fixes, reproducibility via seeded default_rng(7), and 847/847 tests green.

✨ New Features

Lab Guide PDF (the flagship of this release)

• Glossary back matter: 90 alphabetical entries covering tensor/memory, autograd, training systems, architecture, optimization, and ML basics, with module cross-references (see glossary.qmd).
• Module opener hooks: every module starts with a 2–3 sentence systems-first framing paragraph leading with memory, bandwidth, arithmetic intensity, HBM, or roofline implications before the ML story.
• Code listing captions + List of Listings: ~60 substantive code blocks carry Listing N.M — Description captions; populates a new List of Listings in the front matter.
• List of Figures + List of Tables in front matter (151 newly-captioned tables).
• Running headers with chapter + section: verso Chapter N · Title, recto N.M · Section, wordmark centered. H&P / CLRS convention.
• Personal instructor note signed callout at the end of the conclusion.
• Single-source build: make install-deps && make pdf works identically locally and in CI; deps live in pdf/apt-requirements.txt and pdf/tex-requirements.txt.

TinyTorch Framework

• Tensor API expansion: added view(), masked_fill(), Tensor stacking, ndim, numel(), and contiguous() (closes #1298; PR #1392 by @Shashank-Tripathi-07). PyTorch-compat test coverage added for all new methods.
• no_grad() context manager: autograd now supports with no_grad(): inference blocks plus graph cleanup between passes.
• Tito CLI: module path --about renamed to module path --guide and repointed at the Quarto chapter (consistent with the Lab Guide becoming the reference).

🐛 Framework Bug Fixes

Autograd and training

• Tanh wired into enable_autograd() — was silently producing zero gradients.
• Trainer.evaluate accuracy for regression models corrected (was misreporting).
• GELU gradient mismatch + float32 test precision fixed.
• Trainer init: guard requires_grad loop against non-Tensor params; ensure model params have requires_grad=True (2 commits).
• M06 _reduce_broadcast_grad aligned with module conventions.

Module-level

• Quantization (M15): constant tensor quantized to all-zeros, losing the original value (#1444).
• MaxPool2d: API mismatch in milestone 04 CIFAR script fixed (#1278).
• Export paths: corrected for modules 09 and 13.
• Token constants: refactor cleanup from PR #1279 (#1256).
• M19 benchmarking: MLPerf trademark attribution added, educational-purposes disclaimer, table alignment fix, addresses feedback from #1196.
• M02 activations: improved activation graph visualization.

Module audit fixes

• Seven batches of audit fixes landed: batches 1–7 covering critical fixes, medium/low documentation and accuracy, and test-infrastructure cleanup. Final state: 847/847 tests passing.

🔬 Tests

• Finite-difference gradient correctness tests added for Module 06.
• Module 08 training infrastructure coverage tests added.
• Gradient correctness suite restored with per-op tolerances (#1342).
• Module 10 tokenization tests now use real Tensor params instead of raw numpy arrays.
• Module 08 scheduler lr assertion corrected (epoch 0, not 1).

🔧 Engineering

• Reproducibility: migrated from legacy np.random to default_rng(7) — seeded, per-call RNG across all modules.
• Python baseline: minimum version bumped to 3.10. Milestone 05 docs updated.
• src/*/ABOUT.md cleanup: 20 stale duplicates deleted (−20,876 lines); the single-source ABOUT.md now lives in the correct companion-doc location.
• Security: all 28 GitHub code-scanning alerts resolved.
• Tito: register --tinytorch pytest flag in conftest; fix UnicodeDecodeError on Windows in tito module complete (#1184); null-synced_modules guard in submission progress response.

📖 Content Improvements

• Systems-first narrative: every module hook leads with the systems angle (memory, bandwidth, compute, hardware utilization) before pivoting to ML theory.
• Check Your Understanding callouts: converted from prose sections to callout-tip format with 3–5 technical-specific checkboxes per module.
• Key Takeaways: 3–4 bullet recap plus next-module hook at the end of every module chapter.
• Systems Implication callouts unified to callout-note across all 21 instances; answers converted to callout-tip collapse="true" across 101 Q&A pairs.
• Cross-reference audit: 216 orphan table/figure/listing labels got natural prose references (87% coverage).
• Further Reading hyperlinks: 20 external URLs verified and linked (Jay Alammar, arXiv papers, Karpathy's blog, Jurafsky & Martin SLP3).
• Broadcasting pitfall now taught alongside the broadcasting feature (M01 tensor).
• log_softmax implementation cleaned up with clearer variable names and reuse.
• Type hints: added to M03 layers, M04 losses, M05 dataloader (#1167).
• Big-picture diagram: redesigned as a 4-layer stack (Capstone → Optimization → Architecture → Foundation) in neutral palette with MIT-red Capstone accent.

🎨 Design and Typography

• Book-style typography: linestretch: 1.1, first-line indent (parindent: 1.2em), tight parskip. Stripe-Press / Swift-Book density.
• Thin single orange header rule (previously double rule).
• 23 module-diagram SVGs aligned to the book palette via a Gemini multimodal audit pass.
• Arraystretch 1.2 + enumitem for table and list breathing room.
• Text-only callout titles (no stripped emojis; class semantics drive visual distinction).

🐛 Lab Guide Bug Fixes

• Tokenization module: restored missing ```{python} fence that caused Pandoc to render Python variable-definition comments as chapter headings.
• Single-PDF guarantee: Makefile self-heals when Quarto's post-render cleanup strands the artifact at pdf/ instead of pdf/_build/. Build-end banner prints the canonical path.
• 3 broken URLs fixed: GPT-2 cloudfront → OpenAI CDN, PyTorch .md → .html, mlu-explain /relu/ → /neural-networks/.
• Duplicate trailing ## Get Started removed from 4 modules (copy-paste artifact).
• Orphan big-picture-module-flow.svg removed from images/diagrams/ (canonical lives at images/svg/).

🔧 CI / Infrastructure

• Single-source deps: make install-deps reads pdf/apt-requirements.txt and pdf/tex-requirements.txt — same command works locally and in CI.
• tinytorch-build-pdfs.yml and tinytorch-update-pdfs.yml simplified to make install-deps && make pdf (no inline tlmgr package list).
• make clean extended to remove stale *_files/ directories at the Quarto project root.

📚 Documentation

• Early Explorer callout removed from getting-started.qmd — no longer appropriate now that the Lab Guide is shipping.
• Callout convention documented in the preamble: six semantic callout types keyed off Quarto's five shipped classes plus title conventions.
• README tables converted from markdown to HTML format for consistent rendering across GitHub and the Lab Guide.

---

👥 Contributors

Thanks to everyone who contributed to this release:

• @profvjreddi — editorial direction and polish across all 20 modules
• @hzeljko — sustained code, diagram, and infrastructure contributions
• @Shashank-Tripathi-07 — Tensor PyTorch-compat API (ndim, numel, view, contiguous, masked_fill; PR #1392; first-time contributor!)
• @farhan523 — ongoing documentation and module improvements
• @adityamulik — null synced_modules fix in tito submission progress
• @harishb00 — type hints across M03/M04/M05

🆕 New Contributors

• @Shashank-Tripathi-07 made their first code contribution (PR #1392)

---

Full Changelog: tinytorch-v0.1.9...tinytorch-v0.1.10

Website: https://mlsysbook.ai/tinytorch/

PDF: https://mlsysbook.ai/tinytorch/assets/downloads/TinyTorch-Guide.pdf

Glossary (new): https://mlsysbook.ai/tinytorch/glossary.html

MLSys·im 0.1.1

MLSys·im 0.1.1 — Paper Title Correction

First-principles infrastructure modeling for the Machine Learning Systems textbook.

Metadata-only patch release. No code or API changes; safe drop-in
replacement for 0.1.0. Corrects the paper title cited in three places
to match the actual title of the companion paper.

📚 Documentation

• Paper title corrected across CITATION.cff, the BibTeX snippet in
  README.md, and the reference docstring in mlsysim/core/walls.py.
  Was: "A Composable Analytical Framework for Machine Learning Systems."
  Now: "MLSys·im: First-Principles Infrastructure Modeling for Machine
  Learning Systems."

🏗️ Packaging & Dependencies

• Version bumped to 0.1.1 across pyproject.toml, mlsysim/__init__.py,
  and CITATION.cff; date-released updated to 2026-04-24.

Contributors

• @profvjreddi

---

Install

pip install --upgrade mlsysim==0.1.1

---

Links

• PyPI · mlsysim 0.1.1
• Documentation
• Full CHANGELOG
• Diff mlsysim-v0.1.0 → mlsysim-v0.1.1

---

About MLSys·im

MLSys·im is the first-principles infrastructure modeling engine that
produces every quantitative result in the Machine Learning Systems
textbook — memory bandwidth calculations, roofline analyses, TCO projections,
sustainability estimates, and every other number a reader encounters. Each
figure and equation in the textbook is computed, not hand-typed; mlsysim
is the computation.

The framework codifies 22 systems walls — the physical and logical
constraints that bound ML system performance — into composable solvers,
with SI units enforced at runtime. Designed for three audiences: students
building quantitative intuition, instructors running live classroom
demonstrations, and researchers doing rapid what-if analysis.

MLSys·im 0.1.0

MLSys·im 0.1.0 — Initial Release

Release date: 2026-04-01

MLSys·im is a first-principles analytical engine for predicting performance, cost, and carbon footprint of ML systems. It is the computational companion to the Machine Learning Systems textbook.

This is the first public release. The API surface, the 22-wall taxonomy, and the solver portfolio are all considered stable for the 0.1.x line.

Install

pip install mlsysim

Verify the install:

python -c "import mlsysim; print(mlsysim.__version__)"
mlsysim eval Llama3_8B H100 --batch-size 32

Requires Python 3.10+. No GPU required — the engine computes from closed-form equations.

Five-line quickstart

import mlsysim
from mlsysim import Engine

profile = Engine.solve(
    model      = mlsysim.Models.ResNet50,
    hardware   = mlsysim.Hardware.Cloud.A100,
    batch_size = 1,
    precision  = "fp16",
)

print(f"Bottleneck: {profile.bottleneck}")              # → Memory
print(f"Latency:    {profile.latency.to('ms'):~.2f}")   # → 0.54 ms
print(f"Throughput: {profile.throughput:.0f}")          # → 1843 / second

Highlights

Core framework

• 22-wall taxonomy organizing every constraint that bounds ML system performance, across six domains (Node, Data, Algorithm, Fleet, Ops, Analysis).
• 26 analytical solvers (Model, Solver, Optimizer classes) covering all 22 walls.
• Pint unit system with dimensional analysis enforced at runtime.
• TraceableConstant pattern — every default carries a citation.
• Pipeline composer for chaining solvers with explain() and run().
• 3-tier evaluation scorecard: Feasibility → Performance → Macro/Economics.
• Design Space Exploration (DSE) engine with declarative search and constraint evaluation.

Hardware Registry (15+ accelerators)

V100, A100, H100, H200, B200, GB200 NVL72, MI300X, TPUv5p, T4, Cerebras CS-3, Jetson Orin NX, ESP32-S3, nRF52840, Himax WE-I Plus, DGX Spark, MacBook M3 Max, iPhone 15 Pro, Pixel 8.

Full precision support: FP32, TF32, BF16, FP16, FP8, INT8, INT4. Multi-level memory hierarchy with HBM, SRAM, and Flash (TinyML). All specifications verified against manufacturer datasheets.

Model Registry

GPT-2/3/4, Llama-2/3 (7B/8B/70B), BERT Base/Large, ResNet-50, MobileNetV2, AlexNet, Mamba, Stable Diffusion v1.5, DS-CNN, WakeVision. HuggingFace importer included for arbitrary Transformer workloads.

Analytical models

SingleNodeModel, NetworkRooflineModel, EfficiencyModel, ForwardModel, ServingModel, ContinuousBatchingModel, WeightStreamingModel, TailLatencyModel, DataModel, TransformationModel, TopologyModel, ScalingModel, InferenceScalingModel, CompressionModel, DistributedModel, ReliabilityModel, OrchestrationModel, EconomicsModel, SustainabilityModel, CheckpointModel, ResponsibleEngineeringModel, SensitivitySolver, SynthesisSolver, ParallelismOptimizer, BatchingOptimizer, PlacementOptimizer.

CLI

mlsysim eval        Evaluate the analytical physics of an ML system (YAML or CLI flags)
mlsysim serve       Evaluate LLM serving (prefill + decode)
mlsysim optimize    Search the design space for optimal configurations
mlsysim zoo         Explore the built-in registries
mlsysim audit       Profile your local hardware against the Iron Law
mlsysim schema      Export the JSON Schema for the mlsys.yaml configuration file

Testing

367 tests, 100% pass rate. Coverage includes formula unit tests with known answers, full solver suite, physics-bound validation across all registry hardware, wall-taxonomy completeness, pipeline composition, and three optimization backends (exhaustive, OR-tools, scipy).

Documentation

• Site: mlsysbook.ai/mlsysim/
• Tutorials: roofline, memory wall, KV cache, scaling to 1000 GPUs, geography, sensitivity, full-stack audit
• API reference: mlsysbook.ai/mlsysim/api/

Known limitations & gotchas

• First-order analytical model. Predictions are typically within 15–30% of measured throughput on well-optimized workloads. Use MLSys·im to compare options and identify bottlenecks; validate with empirical benchmarks before committing to a production SLA. See accuracy.qmd for full validation against MLPerf v4.0.
• Slide PDFs. Many tutorials cross-link to lecture slides at github.com/harvard-edge/cs249r_book/releases/download/slides-latest/*.pdf. The slides-latest release tag is not yet published; these links will resolve once the slides ship.
• Hosted notebook launchers. Google Colab and Binder buttons are planned but not wired up for 0.1.0. Tutorials run locally on any Python 3.10+ environment.

Project links

• Source: github.com/harvard-edge/cs249r_book/tree/dev/mlsysim
• Issues: github.com/harvard-edge/cs249r_book/issues
• License: Apache-2.0 (code) · CC-BY-NC-SA-4.0 (documentation)
• Citation: see CITATION.cff

TinyTorch v0.1.9 - Computed Values, VS Code Extension & Progressive Disclosure

TinyTorch v0.1.9

Computed values across all ABOUT.md docs, VS Code extension thin client, progressive disclosure improvements, and community contributions.

✨ New Features

• Computed Values in ABOUT.md: Converted all 20 module ABOUT.md files to MyST Markdown Notebooks with inline Python-computed values via {glue:text} — eliminates hardcoded arithmetic errors and ensures all numerical claims are always correct
• VS Code Extension: New thin client architecture over Tito CLI with notebook editor support, build tree, and module explorer
• Version Badge: Auto-updating version badge in site navbar, refreshed on every release via CI

📖 Content Improvements

• Progressive Disclosure: Enforced scaffolding across 9 modules — solution blocks decomposed for pedagogical consistency
• Function Decomposition: Standardized naming conventions and formatting across all 20 modules
• Module 15 (Quantization): Corrected INT8 zero-point values in quantization docs
• Module 16 (Compression): Fixed sparsity percentage bugs
• Module 19 (Benchmarking): Aligned MLPerf box-drawing characters and tree indentation
• EmbeddingBackward: Moved from Module 06 to Module 11 where it belongs conceptually

🐛 Bug Fixes

• Windows Install: Fixed install issues on Windows/Git Bash by @adil-mubashir-ch in #1169
• SocratiQ Typo: Fixed typo in SocratiQ introduction by @BunningsWarehouseOfficial in #1170
• Google Auth iframe: Fixed Google auth and slow index.html loading by @kai4avaya in #1172
• Notebook Filenames: Aligned notebook filenames with Tito convention across all docs (fixes #1176 — thanks @sotoblanco)
• Missing Exports: Added missing #| export directives across 10 modules
• PDF Build: Capped Mermaid figure sizes and fixed nested code fences for LaTeX output
• VS Code Extension: Fixed notebooks opening in raw JSON instead of interactive editor

📚 Documentation

• Updated TITO reference docs to match actual CLI commands
• Fixed broken paths in CONTRIBUTING.md and INSTRUCTOR.md
• Added intra-module scaffolding subsection to progressive disclosure paper

🔧 CI/Infrastructure

• Slide decks download from release during deployment
• VS Code extension artifacts properly gitignored

---

👥 Contributors

Thanks to all contributors who made this release possible:

• @adil-mubashir-ch
• @BunningsWarehouseOfficial
• @kai4avaya
• @sotoblanco
• @harishb00a
• @profvjreddi

🆕 New Contributors

• @adil-mubashir-ch made their first contribution in #1169
• @sotoblanco reported #1176 (notebook filename mismatch)
• @harishb00a contributed documentation improvements

---

Full Changelog: tinytorch-v0.1.8...tinytorch-v0.1.9

Website: https://mlsysbook.ai/tinytorch/

TinyTorch v0.1.8 - Content updates and improvements

TinyTorch v0.1.8

Content updates, website improvements, and community contributions.

✨ New Features

• Team Page: Auto-generated team page from .all-contributorsrc with reorganized Community section
• Slide Viewer: Embedded PDF slide viewer on all module pages for in-browser viewing
• Milestone Visualization: Step-by-step visualization for milestones by @AndreaMattiaGaravagno in #1151
• Site-Only Deploy: New workflow option to deploy website without version bump

🐛 Bug Fixes

• Attention Module: Corrected O(n²) complexity explanation and memory table bug — reported in #1150
• Activations Module: Fixed misleading GELU hint about 1.702 constant — reported in #1154
• Activations Module: Expanded GELU explanation with both approximation forms
• Layers Module: Corrected Xavier/Glorot initialization terminology
• Tito CLI: Resolved Jupyter kernel mismatch causing ModuleNotFoundError (#1147)
• Paper Build: Escaped special LaTeX characters breaking PDF build
• Milestones: Fixed bold cyan frame alignment by @AndreaMattiaGaravagno in #1152
• Content: Fixed small typo by @minhdang26403 in #1163

📚 Documentation

• Specify GenAI usage in slides by @AndreaMattiaGaravagno in #1149
• Added @oscarf189 and @Takosaga as contributors

🔧 CI/Infrastructure

• Download slide decks from release during deployment
• Fixed auto-label permissions for fork PRs (#1153)
• Handle branch names with slashes in fresh install test (#1158)

---

👥 Contributors

Thanks to all contributors who made this release possible:

• @AndreaMattiaGaravagno
• @minhdang26403
• @oscarf189
• @Takosaga
• @profvjreddi

🆕 New Contributors

• @AndreaMattiaGaravagno made their first contribution in #1149
• @minhdang26403 made their first contribution in #1163
• @oscarf189
• @Takosaga

---

Full Changelog: tinytorch-v0.1.7...tinytorch-v0.1.8

Website: https://mlsysbook.ai/tinytorch/

TinyTorch v0.1.7 - Export Reliability Fix

TinyTorch v0.1.7

Critical fix for module exports that were silently failing in CI and some user environments.

🐛 Bug Fixes

• Export System: Uses nbdev Python API instead of CLI for reliable cross-platform exports
• Export System: Fixed directory detection when running from tinytorch/ directory
• Export System: Failures now show full error details for debugging - reported by @lalalostcode in #1146
• Milestones: Fixed Tensor class passing in MLPerf step functions

✨ Improvements

• Paper Link: Now links to arXiv with external link icon (↗) instead of download
• CLI: Invalid commands show helpful error messages

🔧 CI/Infrastructure

• Renamed Publish (Dev) → Preview (Dev) for clearer workflow naming
• All tests run on all platforms by default
• Test types aligned with CLI naming (--user-journey)

---

👥 Contributors

Thanks to all contributors who made this release possible:

• @lalalostcode
• @profvjreddi

---

Full Changelog: tinytorch-v0.1.6...tinytorch-v0.1.7

Website: https://mlsysbook.ai/tinytorch/

TinyTorch v0.1.6 - Windows/Git Bash Support

Windows/Git Bash Support 🪟

The installer script now works on Windows via Git Bash!

Changes

• Platform detection for OS-specific guidance during installation
• More reliable pip invocation using $PYTHON_CMD -m pip
• Cross-platform line endings via .gitattributes
• Virtual environment activation works correctly on Windows

Contributors

Thanks to the community for Windows support:

• @Kobra299 - reported the Windows issue (#1078)
• @rnjema - developed Windows installation improvements (PR #1105)
• @joeswagson - developed PowerShell installer concept (PR #1083)

Installation

Windows (Git Bash):

curl -sSL mlsysbook.ai/tinytorch/install.sh | bash
cd tinytorch
source .venv/Scripts/activate
tito setup

macOS/Linux:

curl -sSL mlsysbook.ai/tinytorch/install.sh | bash
cd tinytorch
source .venv/bin/activate
tito setup

Full Changelog: https://github.com/harvard-edge/cs249r_book/blob/main/tinytorch/CHANGELOG.md