Which AI engineering books should CTOs and technical founders read?

Key facts

• AI Engineering (O'Reilly, December 2024), by Chip Huyen, opens with when and whether to build an AI application on a foundation model before turning to how. (Source: Chip Huyen: AI Engineering book repository)
• Christian Kästner's Machine Learning in Production (MIT Press, 2025) covers requirements, architecture, quality, operations, teams and responsible engineering for products with ML components, and its full text is online. (Source: Christian Kästner: Machine Learning in Production, full text)
• Reliable Machine Learning (O'Reilly, 2022) covers how an organisation should run model development as well as how to operate models reliably. (Source: Google Research: Reliable Machine Learning)
• Introducing MLOps (O'Reilly, 2020) addresses MLOps and model governance at the scale of a whole company. (Source: O'Reilly: Introducing MLOps)
• The Manager's Path (O'Reilly, 2017), by Camille Fournier, follows engineering leadership from tech lead to CTO. (Source: O'Reilly: The Manager's Path)
• An Elegant Puzzle (Stripe Press, 2019), by Will Larson, covers the systems of engineering management: team sizing, planning and organisational design. (Source: Stripe Press: An Elegant Puzzle)

Definitions

Build vs buy

The decision between developing a capability in house, such as training or fine-tuning a model, and paying for one, such as a hosted model API or a vendor product, weighed on cost, control, risk and speed.

Service level objective

A target for how well a service must perform, such as 99.9% of requests answered within a second, used to decide when reliability work takes priority over new features.

Model governance

The policies and records that show which models run where, what data they use, who approved them, how they are monitored and how risk to users is managed.

Why a different list for CTOs

Most AI book lists are written for the engineer who will implement a technique. A CTO's questions come earlier and later: should this be AI at all, which parts should we build or buy, who do we hire, what will it cost at scale, and what happens when it goes wrong in front of customers or a regulator. The books below are grouped by those decisions.

The list also leaves things out. Books that teach one framework's API date too quickly to guide a decision that lasts years, and books on AI strategy without engineering content rarely help with the trade-offs a CTO actually faces: latency against quality, cost against control, speed against risk.

Disclosure: From Model to Product is written by Dr. Dulan Dias, who also writes this site. It appears where it fits, with what it leaves out, and is not ranked above the other books.

Deciding whether and what to build

AI Engineering, by Chip Huyen, is the best current account of what foundation models make possible and what it takes to adapt them, starting with whether to build at all. Read its first chapter and the chapters on evaluation and architecture even if you never write a prompt yourself.

From Model to Product, by this site's author Dr. Dulan Dias, gives two chapters to the step before: framing the business problem and choosing between rules, search, classical ML, deep learning and language models by cost, risk and data. Its premise is that the cheapest reliable approach usually wins. The guide to choosing between rules, ML and an LLM is adapted from it. The question to bring to both books is the same: what is the simplest approach that meets the requirement, and what will it cost when usage grows tenfold?

Running models without surprises

Designing Machine Learning Systems explains what changes once a model meets real data: distribution shift, monitoring and retraining. Reliable Machine Learning adds the operating model, with service level objectives for models, incident response and how to organise model development. Together they answer the question every board eventually asks: how do we know it is still working? For a startup the practical test is simple. If a model's output changed tomorrow, would anyone notice before a customer did? Both books are about making the answer yes.

Teams, governance and responsibility

Christian Kästner's Machine Learning in Production is unusually strong on the parts CTOs own: requirements, the collaboration between data scientists and software engineers, and responsible engineering, including fairness, safety and transparency. Its full text is online, which makes it easy to share with a whole team. Introducing MLOps covers governance once a company runs many models. If you answer to a board or investors, these are also the chapters that prepare you for technical due diligence.

For leadership itself, two general books earn a place: The Manager's Path, by Camille Fournier, on growing from tech lead to CTO, and An Elegant Puzzle, by Will Larson, on sizing teams and designing organisations. The answers on building an AI team and the AI startup CTO role apply them to AI companies.

Architecture, data and cost

Designing Data-Intensive Applications, now in a 2026 second edition by Martin Kleppmann and Chris Riccomini, is the reference for the data systems under every AI product: storage, replication, streams and consistency. It is the book to read before an architecture decision you cannot easily undo.

From Model to Product treats cost as an architectural constraint, with chapters on caching, asynchronous processing, fallback models, load testing and scaling inference, tested on one running product. It is not a management book and does not cover agents.

A reading plan for a new CTO of an AI product

Frequently asked questions

Sources

1. Chip Huyen: AI Engineering book repository
2. Christian Kästner: Machine Learning in Production, full text
3. Google Research: Reliable Machine Learning
4. O'Reilly: Introducing MLOps
5. O'Reilly: The Manager's Path
6. Stripe Press: An Elegant Puzzle
7. Martin Kleppmann: Designing Data-Intensive Applications, second edition
8. From Model to Product, book page