Nov 17, 2013
One of the major goals of dlib is to have documentation that enables
someone to easily make use of its various components. Ideally,
you would read a short description of something, understand it immediately,
and begin using it in your application without any difficulty. Obviously, this
depends partly on the background of the user. For example, if you have
never written C++ software before then it probably isn't going to be this easy.
This page is meant to complement the main library documentation by providing
references to books, along with my commentary, which explain most of
the background material needed to understand the various parts of the library.
In most cases these are the books I learned from during the process
of creating dlib. As always, if you disagree with anything or think I have left out
an important text then shoot me an email.
- Programming: Principles and Practice Using C++ by Bjarne Stroustrup
This is the sort of book you would use in a freshman introduction-to-programming class.
So if you are just beginning to study programming and are interested in C++ then I think
it is probably safe to say this is one of the best books you could read.
- Accelerated C++: Practical Programming by Example by Andrew Koenig and Barbara E. Moo
If you are new to C++ but already know how to program then this is a great book. It's also
about one fourth the size of the Stroustrup book.
- Effective C++: 55 Specific Ways to Improve Your Programs and Designs (3rd Edition) by Scott Meyers
This is a great intermediate level C++ book. Most people have heard the jokes about
how easy it is to shoot yourself in the foot with C++. This book explains many things you
need to know about the language to avoid doing so on a regular basis. So if you are
writing C++ software then this is a must-read. I would even claim that
you are a danger to the C++ software you touch unless you know what is in this book.
I'm not kidding. Finally, the book isn't just about the quirks of C++. It also discusses many general
software engineering ideas which have wide applicability. So in this
respect it is a great book for any software developer to read.
- More Effective C++: 35 New Ways to Improve Your Programs and Designs by Scott Meyers
Consider this an expansion to Effective C++. If you are going to read the above
book then you would almost certainly benefit from reading this one as well.
- The C++ Standard Library: A Tutorial and Reference by Nicolai M. Josuttis
If you are going to buy a reference book on the C++ standard library then this
is the one to get. I think you
will find it is better than any of the available online references. So if you find
yourself frustrated with the online resources, then this is the book for you.
- Online C++ Standard Library Reference
What I said aside, this is a good online reference. I often find myself referring to it
when I do not have the Josuttis book on hand.
- Programming with POSIX Threads by David R. Butenhof
When I was an undergrad, this book was my main resource for learning about multithreading.
It was enjoyable to read, as are all the books on this list, and covered everything
in great depth without becoming overbearing. Also, despite what the title may suggest,
this book is useful for understanding multithreading broadly, not just multithreading
on POSIX systems.
- Unix Network Programming, Volume 1: The Sockets Networking API (3rd Edition)
by W. Richard Stevens
A lot of people call this book the network programming Bible and
this praise is well deserved. If you want a deep understanding of how computer networks
function, including the Internet, then this is the book to read. As with
the Butenhof book above, this is an excellent choice even for people who do not
intend to write software for Unix systems.
It has been a long time since I needed to refer to these two books. However,
they contained information I couldn't find elsewhere no matter
how hard I looked. So I recommend them in case you need to create or understand
some low level win32 code.
- Win32 Programming by Brent E. Rector and Joseph M. Newcomer
- Programming Windows by Charles Petzold
- MSDN Library
This is Microsoft's online reference documentation. It is very large and sometimes
confusing. But at the end of the day you should be able to find the documentation
for just about every function in the entire Windows API.
Computer Science: Algorithms and Data Structures
- Introduction to Algorithms by Cormen, Leiserson, Rivest and Stein
You should get this book if you are looking for a good discussion of the classic computer science
algorithms and data structures (e.g. most of the components on the containers
- Algorithms in C++, Parts 1-4: Fundamentals, Data Structure, Sorting, Searching
(3rd Edition) by Robert Sedgewick
This is another good algorithms book. If you are going to get only one book on this
subject then get the one above. However, when I was learning about these topics I
used both these books and on many occasions I found it helpful to read the description
of an algorithm or data structure in both. Where one description was a little vague or
confusing the other generally filled in the gaps.
Lossless Data Compression
- Text Compression by Bell, Cleary, and Witten
When I was studying data compression this was my most useful
resource. If you are looking to understand how lossless data compression
algorithms work then this is the book you want. It is completely self-contained
and an absolute joy to read. Note that contrary to one of the reviews on
amazon.com, the book Managing Gigabytes is not the second edition of this book;
if this topic interests you then be sure you get the 318 page
book published in 1990.
- Linear Algebra Done Right by Sheldon Jay Axler
If a matrix seems like an arbitrary grid of numbers or you find that
you are confused by vectors, matrices, and the various things
that get done with them then this book will change your whole view of this subject.
It doesn't teach you any algorithms. Instead, it will give you a general
framework in which to think about all this stuff. Once you have that down
everything else will start to make a lot more sense. If all goes well
you will even start to agree with the following: linear algebra is beautiful. :)
- Numerical Linear Algebra by Trefethen and Bau
While Linear Algebra Done Right is fairly abstract, this book by
Trefethen and Bau will
explain some of the actual algorithms that are often used.
This is a great second book if you find that you want to know more about
the SVD, LU decomposition, or various other algorithms involving linear algebra.
- Calculus: Single and Multivariable by Hughes-Hallett, Gleason, and McCallum
Some of the books below will require and understanding of basic calculus. So
I'm recommending this book. It was the book I used as an undergrad and I
remember it being alright. That isn't exactly a glowing review so if you
are really considering buying a calculus book you may want to check out
other reviews before picking this one.
- Introduction to Real Analysis (third edition) by Bartle and Sherbert
At some level real analysis is like a really rigorous repeat of calculus.
So if you already have an undergraduate education in calculus and
you are reading things that seem reminiscent of calculus but involve
stuff you haven't seen before (e.g. sup, inf, "sets of numbers", sequences of points)
then you may be in need of a real analysis book. This one is quite good and should
be accessible to someone with the usual undergraduate computer science math background.
The subject of linear algebra is fundamental to optimization. So you must be familiar
with the contents of a book like Linear Algebra Done Right if you are going to study
this area. You will also need to know how to find the derivative of a function and
understand what a derivative is all about. So you will need to know a little bit of
calculus. Finally, once in a while you will need to know a little bit about real
analysis. Ultimately, what you need all depends on how deep you want to go.
- Practical Methods of Optimization (second edition) by R. Fletcher 1987
I love this book. When I got it I literally spent my weekends sitting around
reading it for hours. It is a fascinating and well written introduction to
the subject of optimization. This has been my most valuable resource for
learning the fundamentals of optimization and I cannot recommend it highly enough.
- Numerical Optimization by Jorge Nocedal and Stephen Wright 2006
This is a more recent text on optimization that is also very good. It
covers many algorithms not covered by the above book.
- Introduction to Derivative-Free Optimization by Conn, Scheinberg, and Vicente
If you want to understand algorithms like BOBYQA
then this is a good recent book on the subject. Note that a book like Practical Methods of Optimization
is almost certainly a prerequisite for reading this book. As an aside, BOBYQA is not discussed in this book but
its predecessor, NEWUOA is.
- Artificial Intelligence: A Modern Approach (3rd Edition) by Stuart Russell and Peter Norvig
This book is about the much broader field of AI but it contains an excellent introduction
to machine learning and it also covers other useful topics like bayesian networks.
Moreover, it is very well written and self-contained. So you don't need any particular
background to be able to learn from it apart from a typical undergraduate background
in computer science.
- Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond
by Bernhard Schlkopf and Alexander J. Smola
Most of the machine learning tools in dlib are implementations of various kernel methods.
So if you want a book that covers this topic in great depth as well as breadth then this is
probably the book for you. The most important prerequisite for this book is linear
algebra. Virtually everything in this book depends on linear algebra in a fundamental way.
The second important subject is optimization. Whenever you see the text
mention the KKT conditions, duality, "primal variables", or quadratic programming it
is talking about ideas from optimization. A good book which will explain all this to you
is Practical Methods of Optimization. Note that this book calls the KKT conditions
just the "KT" conditions. It is talking about the same thing. Also, duality
is something that comes up a lot in optimization but in the context of machine learning
usually people are talking about a particular form known as the Wolfe Dual.
It would also be good (but maybe not critical depending on which parts you want to read) to
be familiar with real analysis.
- Kernel Methods for Pattern Analysis by John Shawe-Taylor and Nello Cristianini
This is another good book about kernel methods. If you have to choose between
this book and Learning with Kernels I would go with Learning with Kernels. However, it is
good to have both since reading different presentations of difficult subjects
usually makes learning them easier.
- Structured Prediction and Learning in Computer Vision by Sebastian Nowozin and Christoph H. Lampert 2011
If you are looking for a book discussing the background material necessary
for understanding things like the Structural SVM
tools in dlib then this is a good book. It is also available online
in PDF form.
- Digital Image Processing by Rafael C. Gonzalez and Richard E. Woods
This is a terrific introduction to digital image processing.
By and large this book doesn't require any special prerequisites. Sometimes
calculus shows up, but not too much.