Book Review: Orchestrating Docker, by Shrikrishna Holla, PacktPub publishing

I just finished reading a review copy of "Orchestrating Docker" a new book from PacktPub, an excellent work from Shrikrishna Holla.  I found the book to be fairly comprehensive about Docker basics, building images, orchestration and even how to build Docker itself, and how to integrate Docker into OpenStack (Havana release via the nova driver).  The book is also very well written, it's a pleasure to read.

That said I do have some criticisms of the book, but the book is still excellent overall.
What is really strange is that the first chapter starts describing how to install Docker (*), and then jumps straight into how to integrate into DevStack (an all-in-one developer version of OpenStack) followed by how to build Docker from source from within a Docker container.

Whilst most books tend to start saying how to install Docker I prefer always to refer to the excellent online documentation, but I guess it's OK to detail this as long as there is a reference to the latest online documentation.  I'd still tend to put this in an appendix though.

Likewise, and even more so covering how to install Docker under DevStack should really be an appendix and should certainly not be in the first chapter before even covering Docker concepts.
Same remark for building Docker under Docker.
I do actually appreciate the DevStack and Docker building instructions but they should be in appendices, in my view.

OK, despite this curious organization of the books chapters the book is still an excellent read and goes into great detail about Docker basics.  Sometimes it feels like a reference manual specifying some docker CLI options without explanation of the terms used.  I've been using Docker for over a year now, nevertheless I did learn a few very useful tidbits from this section (in particularly the difference between load/save, import/export and also between ADD/COPY in Dockerfiles).

Often the "reference" section details a command-line option as 'exists since version 1.2" or similar.
It would be a good idea to state what is the baseline Docker version which is being documented as Docker is moving fast, Docker 1.5 should be out this week.

The book is very complete in the examples provided.  Whilst I've not tested all the examples they seem to make sense and exercise very many of Dockers capabilities of the docker client, the daemon, the REST api, interacting with Docker hub or a private registry, building images, linking containers and so on.

Chapter 3 covers setting resource limits (CPU share, memory usage, disk usage), mounting and sharing volumes,  choosing storage drivers, network configuration, linking containers directly or via the Ambassador pattern.

Chapter 4 covers automation via the REST api, and how to use service discovery with etcd to maintain connectivity across a group of containers dynamically (the Ambassador pattern does this but statically links containers causing the system to malfunction if containers die)

Chapter 4 covers orchestration also, mentioning the newly (December 2014) announced Docker Machine, Docker Swarm and Docker Compose but this was a bit light due to the recent nature of these beta products from Docker.

Chapter 4 terminates discussing security and best practices.
Looks like Chapter 4 could have been split into several chapters, by the way.

Chapter 5 is called "Friends of Docker" and covers Puppet and Chef configuration management tools and the use of apt-cacher.   Dokku is then used as an example (it's a mini-PaaS Heroku clone).

Then there is discussion of high-availability and scaling and examples of using CoreOS/etcd (for service discovery) and Fleet.

Overall this is an excellent book with lots of very useful well written content and is possibly the best Docker book out there right now.  I recommend it.

Video Review: Introduction to Go Programming, by John Graham-Cumming, O’Reilly Media

Exactly what I wanted, a fast-paced introduction to Go and it did the job of getting me started on my first real Go program.  This is an excellent introduction to Go Programming - just what it says on the tin.

This video is not for people new to programming, it’s fast paced and I find that it assumes quite a bit of programming knowledge.  This is not a criticism at all - what makes it unsuitable for complete beginners is exactly what makes it ideal for experienced programmers who just want to get up to speed with a new language.

I was impressed that the author, John Graham-Cunning, does each video episode live, continually writing and re-writing code segments on the fly with hardly any errors or hesitation.  He quickly moves from task to task with just enough explanation so that you can follow making it a very efficient learning process.  There were times when the videos raised questions in my mind, questions I knew I’d have to search for the answer myself.

But again this is fine, if the author provided such a deep analysis as to anticipate all our questions … it would make for a very boring video, which this was not.  The author is also very pleasant to listen to.

The video starts out by covering the basics of the language - the inevitable hello world program, how to build from source and the included godoc documentation tool.  From there we treat different data types (int, string, bool, arrays, slices, maps), multiple return values, and control statements (if, switch, for loop).

After this introduction the concept of concurrency is introduced covering go routines, channels, the select statement and buffered channels.  It was particularly interesting to see how Go handles concurrency.

Next up are interfaces and packages where Go does not follow standard object-oriented practices, probably because of the desire to not implement classes or more specifically to not implement subclassing, the bain of many an OO language.  Go does still provide abstractions for data and behaviour through the struct keyword but it does not instantiate objects or subclass.

This in particular is an area that I’d like to have been able to read something related to the subject.

I found this discussion quite interesting about Go’s approach.

The last 10 or so videos cover some of the practical aspects of Go programming such as the tool chain, unit testing and some of the most frequenly used standard packages for i/o, http, json, strings, sorts, logging, time and how to link with c code.

This is the second video training that i’ve followed from O’Reilly and although I’ve really enjoyed them both and learnt a lot from them I still feel the need for some written support.  Even if it’s some high level slides - actually I’d prefer to have some detailed writing and some high-level slides to accompany the videos - It would really help me get the best from the videos.

Why?

Well I find that my attention sometimes drifts from the videos, and also sometimes I listen to the videos at times when it isn’t practical to watch them.  It would really help me to be able to read the same or similar content and then I could follow more easily when watching/listening.  Of course there are perfectly good documents available on Go, but it would be good to have some referenced text associated with each video.

I was keen to learn more about Go, Googles’ 5 year old systems programming language, having some significant past experience with C/C++ in my early career and still bearing the scars of pointer exceptions and all those weird hoops you have to jump through to get things done in C++.  I’ve also done a bit of C++11 recently which has really advanced the language from the one I knew in the 90’s.

These videos had the effect on me of dispelling Go as some weird language with weird conventions, an idea I’d built up whilst watching some discussions on the Go (Google Groups’) mailing list.  So Go looks pretty impressive to me, only time will tell.  Now I have to continue using it.

I thoroughly recommend these videos for experienced programmers who need something to get them started on Go.  5 hours 17 minutes of video.  Excellent.
The videos can be found on the O'Reilly product page here.

Video Review: Mastering Vim by Damian Conway; O'Reilly Media

I've used vim for 20 years now ... so there's nothing to learn right?
Of course I'm wrong, vim is enormous and these videos do a great job of giving insight into vims' amazing capabilities.  I was under no illusion, I was sure I was only using a small amount of vim/vi capabilities and these videos confirmed that to me.  It was nevertheless a surprise just how much I did learn and I was frequently amazed at functionality I hadn't imagined.

I've always appreciated vims' multiple undo/redo, a vast improvement on vis' binary undo/redo capability, but I was amazed to discover that vim now allows multiple branches of undo/redo and the ability to say go back 10 minutes (through :later and :earlier commands) to whichever branch was being worked on at that moment.

Another wow moment was with vims' visual block editing, allowing to apply changes to a series of lines, moving a column of several lines at once for example.

These videos will take you through a whole range of vim capabilities.  If Damian Conway tells us we've only touched 1% of vims' capabilities, well that's a testament to how extensive vim is.  Yet, he covers many aspects in great detail.

These are not videos that you should just sit and watch, there's too much detail and you won't retain much that way.  Best to occasionally listen to several videos, but when you want to really advance you'll have to take one video at a time either making notes or experimenting.  You'll only advance by working on small chunks ...  downloading into the firmware in your fingers as the speaker tells us.

The videos cover the vim help system ('because you need help'), the various single letter commands,  moving around, search facilities and preview, search and replace, the various undo capabilities and undo history, command and text completion, visual modes and folding text.

Made up of 22 chapters, each chapter will extend your vim usage in many ways, just take your time to learn one chapter at a time.

I appreciate that the speaker provides a tar file of his own .vimrc and various plugins, this will save you a lot of typing.

I regret that there isn't some sort of transcript, or slides available but that's the nature of a video product.
I'd appreciate being able to read the content as well.

Nevertheless, this product is well worth the money and if you invest the time it will pay dividends in productivity.  I will watch the videos again and again ...

Book Review: Test-Driven Development with Python by Harry Percival; O’Reilly Media

This is an incredibly thorough book from Harry Percival.

From start to finish the reader is stepped through the creation of a Python3/Django-based application with great rigour following the clearly described principles and methods of Test-Driven Development as espoused by the (Obey the) Testing Goat.  With this book you will learn TDD but also many tools for Python/Django/Web-development and you’ll be able to reproduce all the code yourself, or obtain the code chapter by chapter from github.

What’s incredible is the number of tools and frameworks which are covered in this book, with such attention to detail on all the installation/configuration/usage steps so that you can follow along.

The very first test you’ll write is a functional test to check that you have Django up and running.  From there on, not a line of functional code is written without it’s test case having been written first, and that’s exactly the way it should be according to TDD philosophy.

The book has been updated to use Django 1.7 - which was in pre-release when the book was written, but has now been released in Sept. 2014.

I recommend a visit to the author’s web site for the book, http://www.obeythetestinggoat.com/, to learn more about the book and even read the book online as a prelude to actually buying the book.

The book covers an extensive list of pre-requisites so that all readers are on the same page before providing an introduction to the concepts of TDD.  The Pre-requisites section includes the use of git, Python3, Django1.7, and reflections on use of Windows and IDEs.

Every code snippet in the book includes the relative path to the file at top right.

This is extremely useful, and makes all the difference, allowing to follow along with the book step by step as we jump from file to file in a well structured application and its’ test code.

I really appreciate that the example souce code is available for the book on github, with a separate branch for each chapter.  I stepped through the book page by page but inevitably made some typing errors, so being able to compare with that chapters’ official code helped me to stay on track.

It was surprising at each step to read “now let’s do xxxx” and then the following code is changes to the tests before doing the declared code change - that’s exactly the way it should be in test-driven development.

It’s impressive how the author advances in a step-wise fashion, often introducing code which is in fact temporary because we’ll later be shown a better way of doing things (e.g. defining the expected html response directly in the test, then modifying the test to refer to the template file contents instead).   This

is extremely powerful as the initial code often introduces useful concepts, the reader is led from the initial solution to a better one thus reinforcing the point - this is very effective teaching!

I should point out that if I am experienced with Linux, Windows, Python, Git I have near to zero experience with Django and several other concepts here.  So I was impressed that despite my knowledge gaps I could follow along hitting exactly the same errors as the author and fixing them exactly as described.

I decided to use my Windows8/Cygwin environment to run through the book, updating my existing Anaconda installation (rather than my Cygwin installation), so I documented how I did this in this blog post.

It’s impressive the attention to detail in this book covering all the practical details needed to reproduce the examples, including anticipating problems with Windows, e.g. with a pointer to pre-compiled Windows binaries for Fabric.

The book is divided into 3 sections.

Part I. The Basics of TDD & Django

This section of the book gets us up and running with the basics of testing a Django application using functional and unit testing following the TDD methodology.

At the end of this section we have a basic application for managing to-do lists using a simple sqlite3 database and a set of unit and functional tests used to check that we can create and consult independent lists.

Use of tests to validate code refactoring.

This alone could have been a book, stepping us through the creation of a Django-based  application for managing to-do lists using a simple sqlite3 database and a set of unit and functional tests used to check that we can create and consult independent lists.  The TDD methodology is clearly described as we write tests before code and use the tests to double check when refactoring code.

Part II. Web Development SIne Qua Nons

The second section treats many topics which would inevitably be encountered in developing, staging and deploying a real web site.  Use of Bootstrap and CSS is covered, then we look at how to produce a staging site, fully tested of course, automating the setup using Fabric, before covering Django’s form classes, duplicate item handling and the use of qunit for Javascript/jQuery testing.

Part III. More Advanced Topics

The third section “Advanced Topics” covers user authentication using Mozillas’ Persona, Mocking in Javascript, Ajax testing, and then Mocking in Python.  Even Continuous Integration using Jenkins is covered.

I thoroughly recommend this book which ably describes the TDD method, but will also step you through the use of many of the useful tools and frameworks a professional developer would use.

Book Review: [Manning] Single Page Web Applications by Michael Mikowski.

I recently had the opportunity to review Manning’s Single Page Web Applications by Michael Mikowski.

I’m impressed by this book.  The book is rigorous and extremely well structured - magnificently orchestrated.  Organized into 3 parts, “Introducing SPAs”, “The SPA client”, “The SPA server”, the author goes to great lengths to provide a very broad coverage of the subject.  Starting with the raison d’etre for SPA’s, then coming back to JavaScript basics to more advanced principles such as anonymous functions and closures before moving on to client-side SPA, then server-side SPA including use of MongoDB.  But the author is also very rigorous taking the time to explain the use of JavaScript prototypes representing a more JavaScript approach to object-oriented programming, as well as code verification using JSLint and testing using nodeunit.

Each Chapter in the book steps you through the creation of a chat application in a very methodical way as new functionality and techniques are added in isolation.  For example, during the ‘SPA client’ part, TaffyDB is proposed as a client-side datastore, which is then replaced by MongoDB in the ‘SPA server’ part.  It’s very useful to introduce TaffyDB, not just to show the possibility but also to allow it’s use for unit testing.

The book lies somewhere between a reference work and a cookbook because despite it’s very methodical tutorial-like approach it lacks some necessary steps to make it a cookbook. These omissions are minor, for example it’s implicit that JSLint can be used to verify JavaScript embedded in html files nevertheless it isn’t described how to do this (or maybe there was a problem with my JSLint installation - attempts as using JSLint on html would complain about ‘<’ Unexpected on line1, i.e. on the first HTML tag seen).

If you want to see what the final application looks like you’ll either have to work through the book cover to cover, or download the source code from Mannings’ site and jump in at chapter 9 - you’d have to know how to launch your application though (not hard, but you’d have to know ‘nodejs ch09/app.js’).

It would be nice to have the code available in working form, as a template in a PaaS (OpenShift for example, or Nitrous.io, Heroku …) and/or as a Docker (trusted build) image.  Having the sources available as git repo would be a good idea.

But I’m being very demanding here … we’re so spoilt in the 201x’s …

I created a Dockerfile available here and intend to make a trusted build available as that’s how I like to work.

The book concludes with Appendices with more than 30 pages on JavaScript coding standards and JSLint, followed by 30 pages on testing with Node.js/jQuery/nodeunit.

I’m no JavaScript expert but I’ve done some JavaScript, REST, MVC, MongoDB.  I found this book very easy to follow as the application is built up.

An impressive work crammed with good teaching and best practices!