Linear Algebra and Optimization with Applications to Machine Learning. Volume I: Linear Algebra for Computer Vision, Robotics, and Machine Learning

The "with applications" in the title of the book should be read as "applicable" because it provides the fundamental mathematics, but it does not explicitly treat the applications that are mentioned. In this volume I, these fundamentals are basically linear algebra, and in volume II it is promised to cover optimization. I can imagine though that there is some interaction like for example the important subject of linear programming. Except for a few illustrative examples, the applications themselves are supposed to be covered in other courses or textbooks. There is a bit of statistics, which I think is also important for the applications mentioned, but probability and statistics as well as calculus is assumed known, but anyway, whatever is needed from these is recalled briefly.

Since this volume introduces the fundamentals with applications in mind, it is in some sense similar to a first course in linear algebra that I have been teaching to engineering students for many years. This volume has also some numerical procedures and even matlab codes. Those I taught in a separate numerical course. The procedures discussed include Gaussian elimination, Cholesky factorization, QR decomposition, eigenvalue and SVD algorithms, Krylov and Lanczos methods, but it skips the basic numerics of rounding errors, error propagation, numerical stability, and the more analytic problems such as numerical quadrature, differential equations, zero finding, etc. The latter also have a definite link with linear algebra, but clearly including all applications of linear algebra is an interminable task.

I wrote my own lecture notes, not satisfied with the existing books that were not providing the desired abstraction and that spent too many glossy pages on the introductory level with many examples and applications. In many ways my notes were very similar to the material covered here, which is why I like this book so much. But since I was trying to cover as much as possible in the most efficient way restricted by the amount of credits that were assigned to the course, my notes were much more concise. This is quite different in this book since, at its introductory level, it is almost encyclopedic and it is like the text I would have liked to write if I were not restricted by a time limit to cover all the material. For example, I spent some time explaining that finite dimensional real vector spaces and linear maps can be treated in an isomorphic way by discussing $\mathbb{R}^n$ and matrix algebra, and then I could just do matrix algebra. Not so in this book. The abstract vector spaces remain present throughout the book. Infinite dimensional vector spaces are a problem because there you need infinite sums and convergence, which require topology to define convergence, and the maps are operators. The authors here maintain some elements of function spaces but certain analysis aspects are not really covered in detail. But otherwise almost all the proofs are fully written out.

Thanks to LaTeX it is nowadays no problem to produce a professionally looking text. The illustrations however require different tools and producing good quality graphics is a challenge. Clearly the authors of this book had the same problem with graphics that I also had. The text is excellent, but the graphics are definitely of lesser quality. Unfortunately it is not only a problem of how they are generated and reproduced, also they do not always make very clear what they are supposed to illustrate.

What would one expect in a basic linear algebra course for engineering-type students? I think that should include vector spaces and linear maps and how they relate to matrices, the rank of a matrix with range and null space, determinants (in my opinion as little as possible), linear systems with Gaussian elimination, normed and Euclidean spaces, orthogonalization and QR, eigenvalue and singular values with generalized inverses and least squares, and geometric interpretation of all these concepts. All this is extensively discussed in this book. The numerical and matlab algorithms and certainly the iterative methods, I would rather expect in a more specialized numerical course, but it is not completely unexpected that they are found here in this book. More unexpected are the following: A discussion about the Haar wavelet with some signal and image processing; the chapter on linear systems is introduced with a discussion about the computation of interpolating Bézier curves; and the computation of the matrix exponential, important for the solution of differential equations, is discussed to some extent. There is also an extensive discussion of groups (SU(2), SO(3), and quaternions. This is important for robotics. Finally, there is much material related to graphs: graph Laplacians, clusters, and graph drawing. The final chapter about polynomial factorization and the Jordan form is less elementary and not always found with the same detail in a basic course. So there is a lot of material, and I can imagine that one wants to make a selection. No advise is given by the authors and it would be difficult anyway since successive chapters rely on previous ones. The authors have earmarked only some sections that they considered to be more advanced and these can be skipped initially.

To conclude, I can definitely recommend this extensive book on linear algebra that is both self contained and thorough and mostly remaining at a basic level. I have always considered linear algebra a basic tool in many applications. The applications mentioned in the title are computer vision, robotics and machine learning but they are not really discussed. However the Haar wavelet is a hint to image and signal processing, robotics are related to the study of the quaternions and the rotation groups, the linear algebra and the graphs are useful for a lot of applications, thus also for machine learning. To come to the actual applications though will need extra material, continuing the basics given here, but also some extra material for example statistics, and optimization (the latter is promised in volume II). Every chapter has a list of exercises (no answers are provided). There is a bibliography which lists mostly books and an index (12 pages). With a book of this size, the index can never be extensive enough. I tried to look up some topics that were not listed. On the other hand, I noted separate entries for "Jordan block" and "Jordan blocks", which makes no sense of course. I know from experience that collecting an index in an artisanal way is, just like generating good graphics, a time consuming task that needs patience and many iterations.

Adhemar Bultheel
Book details

This is a extensive and thorough introduction to linear algebra that includes some extras like wavelets, Bézier curves, groups of rotations, quaternions, and applications in graph theory, that are of particular interest for applications in computer vision, robotics, and machine learning.

Author:  Publisher: 
9789811206399 (hbk), 9789811207716 (pbk), 9789811206412 (ebk)
GBP 175.00 (hbk), GBP 85.00 (pbk), GBP 70.00 (ebk)