AI for Robotics (1 ed)

This book approaches robotics from a deep learning perspective. Artificial intelligence (AI) has transformed many fields, including robotics. This book demonstrates how to reframe decades-old robotics problems as AI challenges and serves as a guide for addressing problems using contemporary techniques in an era of large foundation models.

The book begins with an introduction to general-purpose robotics, how robots are modeled, and how physical intelligence relates to the movement of building artificial general intelligence, while giving you an overview of the current state of the field, its challenges, and where we are headed. The first half of this book explores the definition of problems in robotics, how to frame them as AI problems, and the details of solving them using modern AI techniques. First, we look at robot perception and sensing to understand how robots perceive their environment, and discuss convolutional networks and vision transformers to solve robotics problems such as segmentation, classification, and detection in two and three dimensions. The book then details how to apply large language and multimodal models for robotics and how to adapt them to solve reasoning and robot control. Simulation, localization, mapping, and navigation are framed as deep learning problems and discussed with recent research. Lastly, the first part of this book discusses reinforcement learning and control, and how robots learn via trial and error and self-play.

The second part of this book is concerned with applications of robotics in specialized contexts. You will develop full-stack knowledge by applying the techniques discussed in the first part to real-world use cases. Individual chapters discuss the details of building robots for self-driving, industrial manipulation, and humanoid robots. For each application, you will learn how to design these systems, the prevalent algorithms in research and industry, and how to assess trade-offs for performance and reliability. The book concludes with thoughts on operations, infrastructure, and safety for data-driven robotics, and outlooks for the future of robotics and machine learning.