DockerCon 2017

So much has already been said about DockerCon 2017 and the many announcements so I won't go there, I'm way too late ... but here's a short (oops ! long !!) post on my trip to DockerCon in Austin, TX at the end of April.
So here is a more personal account of DockerCon which for me was a fantastic event full of opportunities.

As mentioned in this "Unikernels, Unikernels, Unikernels ... DockerCon !! post, I really appreciate the support I received from many people starting with my wife Paulina, as it was not in easy circumstances.

As I tweeted I really appreciate the community that Docker Inc. have built around Docker (Moby), Meetups, Docker Captains and they go out of their way to be inclusive - everyone has a role to play and it is valued.

I appreciated the hand-written card I received as a thank-you as a speaker, I was equally impressed to see Docker employees also receive hand-written thank-you cards for their efforts - e.g. Amir for his Moby Mingle contributions.

DockerCon 2015
This was my second DockerCon, having been lucky enough to attend DockerCon 2015 in San Francisco.
That was a fantastic event, that I was so lucky to attend especially as expenses were covered by Docker after winning the "Docker Paris Hackathon" in Dec 2014 with Mario Loriedo - and I should point out that Mario was the brains behind that Hackathon win with "our Sublime Docker" entry.

I really enjoyed DockerCon 2015, able to attend the pre-conference Hackathon (my proposed entry, "The Phedds" from the @PhenomalDocker team, and the resulting hack).  That was a great opportunity to make contacts before the conference and the conference itself was really interesting - my best conference so far.


I mention all this just to emphasize that DockerCon 2017 was even better in my experience crammed full of events and possibilities to meet people and make interesting contacts.

DockerCon 2017 - VIP reception
It all started with a missed connection at Houston which I thought would make me miss the "VIP reception" for speakers on the Monday evening, but thankfully as I dropped my suitcase in my Austin hotel room at 20h35 with only 25 mins left to find the venue and attend I figured it was worth it ... in my quest for contacts in the Docker eco-system ..

Not easy joining a party in full swing but thankfully Victor Coisne the Docker Community Manager saw me and said hi ... just as I was leaving ... that made all the difference, so I stayed a while meeting a few people such as Docker Captain Dan Finneran of HPE - we work in neighbouring teams, Mano Marks and several others.  A great start to DockerCon.

Before DockerCon 2017 - I can't sleep so I have to run ...

I love to run or cycle when I travel ... usually cycling is impractical (though I remember a crazy ride downtown HongKong and then around the Island one time ... but it had taken a while to find somewhere to rent a bike).

I love discovering a place like that and now I use runtastic to capture the experience, it's nice to browse through my runs seeing where I've been ... the last 4 runs have been Austin, Montreal, Madrid and Grenoble ...

Jet-lagged, I went out for a run at 6am.  It was pretty dark even on the way back before 7am so it was a little dangerous in the park - almost fell going down a step I didn't see.  I was amazed at the number of runners I saw, some of them sensibly running with headlamps strapped to their forehead.

DockerCon 2017 - Moby Mingles
Docker introduced the concept of Moby Mingles for this conference allowing anybody to propose a subject to discuss in 1-1 meetings.  I would have proposed something in Unikernels but of course Amir of the MirageOS team had already done that.

There were many interesting subjects proposed but the three I was really interested in were the three proposed by Amir: Unikernels, FaaS (functions as a service) and IoT.  In the end I settled for Uniikernels as this was a golden opportunity to exchange with Amir prior to my presentaton the next day.  He invited Johnny Mkhael along also, this was great as Johnny and I had some interesting talks and as he is Paris based we hope to meet in the future - especially as he lives in the same town as my Chilean sister in law, Marcela.

Johnny impressed me by his blogging on http://jmkhael.io/ straight away at DockerCon, I could learn something from him ;-)

DockerCon 2017 - the conference ...
Then of course the conference itself started with the keynotes.

I didn't intend to attend many of the presentations as I figured it would be better to spend time either meeting people, looking at the booths and preparing my own presentation !

I did attend both days keynotes which I really enjoyed - for me the announcement of LinuxKit was really important.

I also attended Abby Fullers (Amazon Developer Evangelist) "Creating Effective Images" and Brendan Greggs (Netflix) excellent "Container Performance Analysis" talks.

DockerCon 2017 - My Unikernels presentation
I was surprised to discover that the 20 minute "community" presentations would be in part of the exhibition area rather than in a separate room.  Learning this I immediately decided to not attend any of those sessions feeling that they would be difficult to follow because of the background noise of the exhibition area.  Nevertheless as a speaker I didn't have any problem and felt at ease presenting to a sea of faces and I didn't even think to estimate how many people were attending ... 100 maybe?

I'd looked at the agenda and decided I could overrun my 20 min slot if needed which I did, though when I stopped at 29 mins and opened for questions they cut the mike ;-)  Nevertheless, I was pleased that several people remained and we discussed for 15 to 20 mins.

I was very pleased over the following days that several people came up to me with positive comments on my presentation - I think this is more that there is a thirst for information on Unikernels today - and I had the chance to meet more MirageOS team members such as Mindy Preston and Heidi Howard (a string believer in Unikernels - a Univ. of Cambridge post doc. rather than MirageOS team member i think).

I was also contacted by Per Uber the CEO of IncludeOS who wanted to discuss about their product.  We had some very interesting discussion.  IncludeOS is not a legacy OS as shown in my slides - not a very wise move calling any active project "legacy" of course.

DockerCon 2017 - the exhibition area

After my talk I was pretty pleased to be able to unwind a little and spent some time browsing the different booths, collecting a few t-shirts along the way.

Actually, my favourite t-shirt was the HPE/Docker t-shirt for it's very quiet tone ...  black t-shirt with small HPE and Docker logos makes this quite a wearable t-shirt - not just another pyjama top !

DockerCon 2017 - The internals summit

The internals summit was held on the Thursday and was a great chance to meet Docker employees, Docker captains and organizers and others.  The goal was to learn about specific Docker, or should I say Moby projects ...

In the afternoon we had different breakout sessions and so I chose to join the Unikernels group initially where we had very interesting disucssions lead by Amir Chaudhary.
I really appreciate how Amir is making efforts to federate the Unikernels community across diverse technology implementations.

I was also very pleased to get the chance to meet Matt Bajor, who's presentation "Look Ma, no OS!" I'd watched and briefly Idit Levine creator of the Unik project.

I later attended the LinuxKit/Infrakit session but joined the session a little late.

DockerCon 2017 - The organizer summit (in parallel with the captain's summit)

On the Friday morning there was an organizer summit, a great opportunity to meet other organizers and discuss what works, what doesn't in organizing meetups and to here from Karen Bazja, Lisa McNicol and Victor Coisne about plans for future organizer community events.

It was appreciated that the meeting ended with a lunch which we shared with the Docker Captains ... a great way to meet more people, in particular I finally got to chat with Alex Ellis.

The journey home
Unfortunately a delayed flight from Newark to Montreal meant an unplanned stopover in Montreal.
The good news was that I managed to change the flight AirCanada were proposing (from a 3 flight periple Montreal-Toronto-Munich-Lyon taking 15 hr to a direct 7hr flight to Geneva) allowing me to have a nearly full day in Montreal before leaving.

Not wanting to take too many risks I headed to Mont Royal and had a nice run before wisely heading back to the hotel to get my bags and off to the airport.

A panoramic view from Mont Royal

I wouldn't have chosen the delay, but this was a nice few hours recompensation ...

Unikernels, Unikernels, Unikernels ... DockerCon !!!

Cutting to rhe chase ... my DockerCon Unikernels talk video and slides are now available

It's the end of a series for me, a fairly tough 6 months of working holidays and weekends to advance on Container Orchestration and Unikernels by presenting in conferences, so finally a blog post or two.

After our Container Orchestration and labs (with Haikel Guemar, Mario Loriedo) at LinuxCon/ContainerCon in Berlin last October I decided to investigate Unikernels.

I was intrigued by reports of "7 Unikernel Projects to Take On Docker in 2015",
and then Dockers' "purchase of Unikernel Systems" in Jan 2016,
closely followed by Brian Cantrills' post "Unikernels are unfit for production"
and very interesting discussions on "Hacker News".

Bryan can be deliciously outspoken by the way ...

The culmination of this initial investigation was being able to present at DockerCon 2017 in Austin in April.
That was such an amazing opportunity for me and I have to say a big thanks to those who supported me on this (My long suffering wife, Paulina, of course !! Christian Schutz, Bruno Cornec, Patrick Masse at HPE, Jenny Burcio and Victor Coisne at Docker Inc) and thanks also to Amir Chaudhry, Mindy Preston and Heidi Howard of the MirageOS team for their encouragement at the event.

The talk video and slides are now available

There were a few steps along the road to DockerCon ...

From that initial interest in Unikernels I decided to respond to a few call for papers and was initially accepted at DevConf.cz, so to warm up I did a  "Docker Grenoble" Meetup to prepare.  Thanks to those who participated to that "alpha release" presentation ... and for the great comments and questions.

Slides at https://www.slideshare.net/MichaelBright3/2017-jan19-meetupunikernels.


Shortly after came DevConf.czhttps://devconf.cz/, at Brno in the Czech Republic close to RedHat's European center of engineering.  This was my first, but hopefully not last time attending or presenting at DevConf.cz.  The first day started with Mario and I running our Container Orchestration lab.  On the morning of my Unikernels presentation I was preparing in my hotel room when I started to see tweets about a Unikernels debate between Steve Pousty "OpenShift Developer Advocate" and Dan Walsh "Mr. SELinux".  The tweets had me thinking there was quite an anti-Unikernel sentiment at the conference which was a little worrying - in fact I've since watched the video and seen that Steve brilliantly argued that Unikernels are worth investigating ...

Slides at https://www.slideshare.net/MichaelBright3/2017-jan29-devconfczunikernels

I had the chance to present on NFV, and Docker tools presentations and Docker 101 ad Orchestration labs with Bruno Cornec at HPE TSS in Cannes in February ... but no Unikernels.  I was amazed one afternoon to check my e-mail to see a mail from Jenny Burcio and Victor Coisne telling me I'd been accepted to give a talk at DockerCon ... very important to me !

I scurried back to Grenoble on the Thursday evening just in time to be able to present on Unikernels at Snowcamp.iohttps://snowcamp.io/2017/en/welcome in Grenoble.  Thanks to the Snowcamp team for that opportunity.

Slides at here https://www.slideshare.net/MichaelBright3/2017-feb10-snowcampiounikernels


In March HPE TES (aka "HPE High Performance Computing and Open Source Linux Technical Excellence Symposium 2017") I had the chance to present on Unikernels,

The official slides are not available, but were based on this slide set on github

So after a series of quite different presentations, some with no demo, some with one demo and the Snowcamp.io presentation where I feverishly created some demos whilst watching the previous speaker session .... I was ready to present at DockerCon.

Despite the initial intention to show several Unikernel technologies, I decided that this wouldn't work in my 20 min slot (well no one was after me so I stole 29 mins in the end ...).

I decided to give an overview of the technologies and follow this with a demo of an example MirageOS unikernel for a DNS server (a modified version of the example https://github.com/mirage/mirage-skeleton/tree/master/applications/dns) showing
- how "mirage configure -t unix" could be used to build a standard Linux executable of the application
- how "mirage configure -t ukvm" could be used to build a bootable unikernel image (no OS folks !) of the application
- running the unikernel locally under ukvm
- running the unikernel under Google Compute Engine

Note: There is no currently available IaaS which is ideally suited to running Unikernels, with GCE it was necessary to create an image with 1Gby disk for example, but the point was to show that the Unikernel technology is there even if the IaaS aren't ready yet.

That's it folks, thanks to all those who contributed in different ways, it's really appreciated !

Next up is a Container Orchestration lab with Haikel, and a Jupyter-in-depth lab with Matthias Buissonier of the Jupyter core team at PyConUS next week in Portland, Oregon.

After that a bit of rest before reflecting on how to get to DockerCon Europe ...

The Art of the Command Line - ls (seen from Jupyter ... seen from MyBinder)

In the spirit of the github repo "the art of the command-line", or Chris Allen's cheatsheets I wanted to put together some cheat sheets and notes about GNU, unix, linux utilities.

But I wanted to produce them using the bash_kernel of the Jupyter notebook thereby allowing to have a notebook representation of the cheat sheet in an executable form (providing Jupyter+bash_kernel are available of course).  I'll put these cheatsheets on github, in a fork of the art-of-the-command-line, here.

In an earlier draft of this blog post I wrote

Ideally, I'd like to point you to an online executeable version of these cheatsheets in the same way we can reference tmpnb, tryjupyter.org or binder.org notebooks but unfortunately none of them provide a bash_kernel.  At least on github you can view them, and also clone them to try out locally.

Thankfully I realized today that this is not the case.
Thanks to Docker, thanks to the Binder project (https://mybinder.org) you can run such notebooks online.

By clicking on the link at the bottom of this page you will launch a container on the mybinder site prepopulated with my ls tutorial notebook in a form that you can modify.

So here's a very first shot at that looking at that venerable performer 'ls'.  OK, not the most exciting example for the moment but more will come, especially combinations of tools.
But why ls?

Well many moons ... many many many moons ago actually, like in the 80s/90s I would look at some HP-UX pages and be surprised - I didn't know about that option!  So I vowed to periodically return to man pages and have another look.  Of course 20-30 years on I don't do that as there are so many commands ... but I should ... sometimes at least.

So I figured that probably even ls has a few surprises - like it's only very recent that I discovered the '-h' human readable option on ls, du and even sort (unfortunately OS-X seems to miss this option on sort).  So as an example on Linux, or Windows/Cygwin, instead of looking at disk usage with
    du -s * | sort -n
I now do
    du -sh * | sort -h
which will show disk space in human readable form and sorted, e.g.
    10k tmp/
    2.1M src/
    1.5G Downloads/

Anyway, you can run the actual notebook tutorial here

In later posts I produce some more exciting tutorial notebooks, and I'll also talk more about Binder.
I have a lot of plans for Jupyter, Binder, Docker, ...

What did you Expect - a look at Python module Pexpect

On Tuesday I presented on Pexpect at the Grenoble Python User Group, a short presentation as Rene Ribaud was presenting DocOpt and staticjinja.

Pexpect is a pure-Python module implementing Expect functionality.  Expect is functionality to enable to automate interactions with some interactive process where the user is obliged to interact with the program.  Expect was originally written in tcl but versions have since appeared in many other languages.

When using expect you spawn a new process and then interact with that process through a series of interactions where the expect() is called and blocks until a suitable match occurs, e.g. a password prompt, and then send() is used to send text to the process as if sent by the user typing at the keyboard.

I started using pexpect because I had a special use case.  When installing a particular N-node OpenStack platform it was necessary to pass by several machines/accounts to reach the controller or compute nodes.  As machines and user accounts were automatically created with unknown passwords and installed ssh keys, it was necessary to perform a sequence of 3 ssh hops and 3 sudo calls to reach a compute or controller node.

Using pexpect it was possible to auto-login across those 6 accounts, by sending the password when necessary.  A script was written mhop_ssh.py to perform such multi-hop logins and even to install ssh keys along the way thus allowing traditional direct ssh connection to the controller or compute nodes.

I've placed my presentation notebook and the mhop_ssh.py script on github here.

For testing purposes I created a group of virtual servers using Docker, described here.

Creating serverless virtual servers with Docker

Well not serverless exactly as serverless is a popular term referring to the deploying of code in the cloud which does not require a dedicated server, as the AWS Lambda service or Google App Engine (GAE) for example.

In this case I needed some ssh addressable servers for testing my Pexpect scripts.  It occured to me that rather than using physical servers, or even creating virtual servers using Vagrant/VirtualBox I could create some Docker containers configured with openssh to act as if they were physical servers.

Whilst not the recommended use of Docker - where typically one or more application processes are created in a container - where "docker exec" rather than ssh would be used if there were a need to connect to the container, it fits perfectly my test use case.

I've placed my Dockerfile in this github repo.
It is used to create an container image running OpenSSH, and with a user 'user' and password 'password' - clearly security was not a concern for my testing !!

I intend to develop this repo making use of more Docker features to allow more complete server/network emulation for tests, demos, labs, presentations.

Creating a Docker Build Lab with Jupyter

In keeping with my "Jupyter with everything" moniker (not to mention Docker, Docker, Docker), I recently had the opportunity to create and run a Docker Build Lab with about 20 participants.

To actually create the Docker Lab I used Jupyter with the bash_kernel to create and document the examples.

Using Jupyter was extremely useful - especially when ill, sat in a hotel room finishing the lab (work was crazy in the weeks before the conference).  It becomes extremely easy and fast to rerun the entire notebook to be sure that everything works and recreate a printable document for use.

Although the lab execution was performed in RHEL7.1 virtual machines, I prepared the lab on a Windows PC running a Ubuntu-LTS via Vagrant.  This worked perfectly - though some students were confused to see mention of 'vagrant' user within the lab document (I'll have to be a bit more rigourous on that next time).

I had some small problems creating a lab document which I wanted to be able to distribute in class as a PDF..
I had trouble producing a PDF (well I could just print the notebook in Jupyter but the stylesheet was causing this to print without colour which made it less readable - hacked around this to print to PDF).
My hack worked and so I could distribute a colour PDF.

I decided to archive to github the Jupyter notebook (.ipynb) also exported as HTML and markdown(.md) aswell as the PDF.

The lab is here https://github.com/mjbright/DOCKER_LABS.
This is actually a clone of the repo I used (whence some links may go to the original version) as I want to work this lab as a series of labs in future conferences.

This was definitely a beta run.

I'd like to break this lab down into several labs
- Docker build essentials: focussing more on the low-level details
- Building an application: going step by step to building an orchestrated solution

Hopefully I'll get the chance to rerun this at MixIT here in France in April, or at HPE TSS in June.

Suggestions for improvement are more than welcome.

Having my cake and eating it ... OSMC (Kodi on Raspberry Pi) and Hypriot/Docker

I was really pleased when Hypriot started producing images to run Docker on the Raspberry Pi.

Docker
In case you don't know Docker which provides amongst other things great application portability only runs natively on Linux/x86-64 ... (OK, let's ignore that Microsoft have ported Docker to their Technical Preview of Windows Server 2016 - for the moment the only official deployable Docker runs on Linux/x86-64).

Hypriot's Docker port
But there have been several ports of Docker to other architectures such as the ARM-processor based Raspberry Pi.  Hypriot have been producing SD-card images for a while now to allow to install/run a specific distribution including their build of Docker.  I had a chance meeting with Dieter Reuter of Hypriot at DockerCon SF in June ... I was surprised to find that they're a group of volunteers in Germany building these images in their spare time.  They've also been gradually including other components of the Docker ecosystem on Raspberry Pi such as Docker Machine, Docker Compose and Docker Swarm.

You can read more about their latest release 'Will' on their blog here 'HypriotOS Reloaded'

OSMC
I also run media software on my Raspberry Pi in the form of OSMC (a Raspberry Pi specific derivation of Kodi - previously known as XBMC or XBox Media Center).  OSMC is written by Sam Nazarko.

Choosing between my media center and my Docker demo
But a man only has so many Raspberry Pi's and so I kept switching between my media center software OSMC and Hypriot's Docker which I sometimes used for Docker demos as part of an OpenStack training at work.

So a month or so ago I posted on the OSMC.tv site asking if it was possible to add Docker support.
As my "media center" Raspberry Pi is also used for some web-scraping cron jobs (using my Not invented here scraper) I was hoping I could use the same machine for my media center, docker demos or other experiments and for my web-scraping (which would be run under Docker ... why not!).

I was very pleasantly surprised by the very quick reply from Sam suggesting he should be able to include kernel support for Docker in the OSMC September release.

OSMC and Hypriot's Docker
So today I realized that OSMC does not include kernel support for Docker.
This evening I downloaded the latest OSMC release and wrote that to a uSDHC card, booted up and then installed the Hypriot Docker 1.8.2 Debian package available from their download page.
Without more a do I had Docker up and running on my OSMC.
To check I pulled and ran the Hypriot supplied hypriot/rpi-node image from DockerHub ... success!

THANKS HYPRIOT THANKS SAM NAZARKO (OSMC) !!!!

Installation instructions

Here are some fairly rough instructions of what I did, which can be resumed as
- Perform a standard OSMC install to a uSDHC card
- Install the Hypriot Docker debian package

To start with I downloaded the latest OSMC release (2015.09-2 for RaspberryPi 2) from their download page.
As I'm running Windows I used Win32 Disk Imager to write to a 32GBy uSDHC card (you can get away with much smaller ~ 2 or 4GBy I think).

I then booted the Raspberry.
On reboot I
- logged in as user osmc (with password osmc)
- added a new user, and installedan  ssh-key to the Raspberry using ssh-copy-id from my Windows machine (I run cygwin).
- Still as osmc I gave sudo permissions to my new user by adding a line to /etc/sudoers
    sudo vi /etc/sudoers
    Adding the line
        user     ALL=(ALL:ALL) ALL
-  I also edited /etc/group to add my user as a member of group docker.
        docker:x:113:user
- I then rebooted

On reboot I performed
    docker ps
to see that I could communicate correctly with the Docker daemon:
    $ docker ps
    CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS  NAMES

This shows that we can communicate with the Docker daemon and that we have no containers running.

I then ran one of the Hypriot Docker images from the Docker Hub and landed in a shell within the container.

    $ docker run -it hypriot/rpi-node bash
    root@a555f5cb2166:/#

Success!

Next step install some images (see below) and start playing ...

Downloads
You can download Hypriots SD-card images for Raspberry Pi, or their individual debian packages here:  http://blog.hypriot.com/downloads/

You can download OSMC from the download page here
    https://osmc.tv/download/images/
Currently it's the 2015.09-2 image which you want.

Running Hypriot provided Docker images for the Raspberry
Hypriot have provided several Docker images on the DockerHub for the Raspberry Pi.

Note: As this is an ARM architecture we require specific ARM architecture images.

We can search for Hypriot images on the Docker Hub as follows:
$ docker search hypriot/rpi
NAME                           DESCRIPTION                                     STARS     OFFICIAL   AUTOMATED
hypriot/rpi-node               RPi-compatible Docker Image with Node.js        24
hypriot/rpi-java               RPi-compatible Docker Image with Java           13
hypriot/rpi-python             RPi-compatible Docker Image with Python         12
hypriot/rpi-iojs               RPi-compatible Docker Image with io.js          11
hypriot/rpi-golang                                                             9
hypriot/rpi-gogs-raspbian      Raspberry Pi compatible Docker Image with ...   7
hypriot/rpi-gpio               Dockerized gpio binary of wiringPi for the...   6
hypriot/rpi-busybox-httpd      Raspberry Pi compatible Docker Image with ...   6
hypriot/rpi-ruby               RPi-compatible Docker Image with Ruby           5
hypriot/rpi-mysql              RPi-compatible Docker Image with Mysql          4
hypriot/rpi-hugo               Raspberry Pi compatible Docker Image with ...   4
hypriot/rpi-drone              Raspberry Pi compatible Docker Image with ...   3
hypriot/rpi-crate              RPi-compatible Docker Image with Crate.io       3
hypriot/rpi-swarm              Raspberry Pi compatible Docker image with ...   3
hypriot/rpi-haproxy            Haproxy Dockerfile                              2
hypriot/rpi-redis              Raspberry Pi compatible redis image             2
hypriot/rpi-alpine-scratch     Raspberry Pi compatible Docker Image with ...   2
hypriot/rpi-gogs-alpine        Raspberry Pi compatible Docker Image with ...   1
hypriot/rpi-tor-relay                                                          1
np83/rpi2-hypriot-nodejs       Raspberry PI 2 can run hypriot Docker host...   0                    [OK]
gdelpu/rpi-hypriot-node-hapi   NodeJs with HapiJs framework                    0
hypriot/rpi-nano-httpd         Raspberry Pi nano sized HTTP server             0
hypriot/rpi-skydns                                                             0

Lines above commencing with hypriot/ are hypriot provided images.

What next?
I saw a post somewhere of someone running Jupyter on the Raspberry Pi ... I must give that a try ...

Jupyter - an educational tool - for demonstrating or teaching about Docker

On Friday I did an N-th presentation and demo of Docker at work as part of an OpenStack training.

Whilst the slides were in Powerpoint-* the demo was conducted principally using Jupyter, the multi-language notebook tool derived from IPython.

* -  this time, but I plan to use Jupyter/RISE for an integrated presentation/demo next time.

As I'd presented Jupyter at a Meetup of our local Grenoble Python User Group on Thursday, and I'm preparing to present at Pyconfr in Pau on October 17th (French) I was eager to really apply Jupyter for some educational use.

Note: I've also submitted a proposal for Confoo in Montreal next February ... you can vote for that talk here (but you'll need to register)

I was also keen to test new formats for presenting.
On Thursday I'd successfully mixed presentation with demo - using the excellent Jupyter/RISE extension from Damian Avila - allowing me to execute code within the presentation slides themselves in the browser.
I find this very powerful and it's something I felt I really needed for my Docker demos.

So this time I didn't have time to redo my slideset, but I did have time - a morning - to put together a demo environment.

When I have more time I'll extend that work to be a much more comprehensive demo.  I've quite a lot of ideas for that around basic Docker capabilities but also other tools such as Machine, Compose, Swarm ... and more ... but more on that in a later post ...

Here's a screenshot of just one part of the demo.
We can see text describing what's happening (this is entered as Markdown), followed by the input cell where we enter real commands, followed by the output from the command.

So this allows to demonstrate and document live coding and later publish the result -as the above Jupyter notebook posted on github.

Here's the notebook showing the demo steps, presented as a narrative thanks to the notebook format.  You can view directly the notebook rendered on github here:

2015-Sep-25_Docker_Demo/2015-Sep-25_Docker_Demo.ipynb

or if you want to run the notebook in a suitable environment you can download the raw notebook here:

https://raw.githubusercontent.com/mjbright/jupyter_notebooks/master/2015-Sep-25_Docker_Demo/2015-Sep-25_Docker_Demo.ipynb

To use that notebook you'll need to have Docker installed and Jupyter.
I don't intend to describe the intricacies of installing Docker and Jupyter on various systems, there are many articles available on this.  I do just provide an overview below.

This was the first post on using Jupyter to demo Docker, there will be many more to follow as I evangelize both tools at work and in conferences.

---

Setup Notes

For Docker you'll need to install directly on a Linux machine or if on Windows or Mac OSX you'll need to use a virtualized Linux.  You can use the Docker Toolbox (basically a combination of Kitematic GUI, VirtualBox and a linux image).

My demo was performed on a Windows machine.
I personally prefer to have more control over my virtual machines, and so I use Vagrant to install my virtual machine running under VirtualBox.
The Vagrantfile I used is available here.

I also used Jupyter to run/record the demo.
The best way to run Jupyter on either Windows or Linux is to install the Anaconda distribution, available from Continuum Analytics here.

You can find out more about the Jupyter project here, or try it out online here

#DockerCon Hackathon, San Francisco – the Phedds are off !

On Saturday morning the #DockerCon Hackathon started prior to the San Francisco DockerCon.

I formed the team « The Phedds » to work on the @fenomenaldocker, « Phenomenal Docker demos » project.  We have a strong team ranging from UI/UX,  Javascript/Java development through to system and of course container skills in our 6 man team.

I’d posted here on this blog and on the Meetup page my idea for this Docker demo system.  Several people showed their interest and so we were able to form a 5 man team.

Our team achieved some awesome work in just 24 hours, thanks to

-       Mario who works for Zenika in Paris, as a Docker-certified consultant/trainer

-       Kalyan Thatikonda who works for Salesforce

-       Zorigt

-       Ryan Van Niekerk, working with GoPro, living working a stones’ throw from the Marriot where the Hackathon is being held.

I was very pleased to have Mario onboard as we had won the Paris Hackathon together with the Sublime-Docker project (Marios’ idea of an integration of Sublime Text editor with Docker).  Mario subsequently created the umbrella DOMIDE project in which he recently added an Eclipse-Docker integration.

I must give a special shoutout to Ryan who had already been thinking about this sort of project for sometime and especially had the skills to implement it.

Ryan couldn’t be stopped as he hacked through the night on VisJS, D3js, Express, WebSockets – he was a man with a mission.

There’s a cool demo video of the project on YouTube here.

I think you can see that there’s real potential for a slick and functional demo system here and we will go further … Ryan is already in the starting blocks for that ...

Whilst we didn’t win the Hackathon, there were 40 teams competing, with some great results – I’ll leave Docker do an official post on the Hackathon results.

Thanks to the while team for their great work !!

I hope the whole team will continue to contribute to the project.

Thanks also to the Docker team and sponsor Rancher Labs for organizing the Hackathon.

Docker Con Hackathon Ideas - #1 "The Phedds" - The Phenomenal Docker Demos

Next weekend I'll be attending the DockerCon Hackathon in San Francisco, just before DockerCon.

We've been invited to propose projects in advance, so here goes with my first and favourite idea.

The idea behind the "Phedds" is to produce a set of ultimate Docker demos.
I've presented Docker many times at work and have generally started with a command-line demo where flipping between showing the basic "docker run" or other commands and then to another terminal to show what's happening in the background, OR I'm showing a Docker build, OR I'm showing docker compose being used to build a cluster of components and then hitting F5 to show that we're reaching different docker containers through load-balancing.

That's all great but what if I could have a multi-paned browser page showing different aspects all concurrently.

Here's  mockup of what that could look like, with a command-line pane top-left (maybe simulated) in some way supplemented by other panes showing some element of resource usage or a dashboard such as Seagull.  Here classical resource widgets are shown, but they could be graphical process tress, or a disk usage tree allowing to show the file system usage and diffs (docker diff information) to show clearly what happening under the bonnet.

I imagine a dynamic visualisation using a library such as D3.js.

Another more specific scenario could be an N-tier architecture, such as a front-end load-balancer, with several web-servers behind and possibly a DB backend.  One pane could show the dashboard of the load-balancer admin page (ha-proxy in this example), another pane could show the current output of the web front-end (or as shown here, several successive views showing the differences as different backend web servers serve up the content).  Another pane might show resource utilisation, whilst another pane might show the architecture ... or any other related "presentation" material.

This final scenario is just to give the idea of having a group of servers in different containers each serving up some interesting dynamic content.  Maybe this could be a group of N, e.g. 8 servers, quickly launched and panes being created as the servers come quickly online.

And this just to remind that all these panes could be highly dynamic in nature to show that high performance graphics and snazzy demos are possible !!

 Whilst the mockups above are all 2x2 panes, it could be possible for each demo scenario to have a template specifying the number of panes and what widget would be available in that pane.


There you go, a quick mockup ... thanks to let me know what you think.

My thoughts for implementing this at the Hackathon:

Rule#1 - challenge these ideas, let's brainstorm !
Rule#2 - this idea could evolve into the phenomenal Docker lab, or a real tool, let's brainstorm !
Rule#3 - let's brainstorm !
Rule#4 - there is no rule#4

Any skills welcome, but at least one or two Javascript and even D3.js hackers are needed to really produce this demo.

NOTE: Although a demo system is proposed here, I'm open.
             If we produce the "best Docker lab" then I'd be just as happy.
             If instead we produce some useful tools, great.
             If we produce the best "bread-slicing container system" why not ...

Book Review: Orchestrating Docker, by Shrikrishna Holla, PacktPub publishing

I just finished reading a review copy of "Orchestrating Docker" a new book from PacktPub, an excellent work from Shrikrishna Holla.  I found the book to be fairly comprehensive about Docker basics, building images, orchestration and even how to build Docker itself, and how to integrate Docker into OpenStack (Havana release via the nova driver).  The book is also very well written, it's a pleasure to read.

That said I do have some criticisms of the book, but the book is still excellent overall.
What is really strange is that the first chapter starts describing how to install Docker (*), and then jumps straight into how to integrate into DevStack (an all-in-one developer version of OpenStack) followed by how to build Docker from source from within a Docker container.

Whilst most books tend to start saying how to install Docker I prefer always to refer to the excellent online documentation, but I guess it's OK to detail this as long as there is a reference to the latest online documentation.  I'd still tend to put this in an appendix though.

Likewise, and even more so covering how to install Docker under DevStack should really be an appendix and should certainly not be in the first chapter before even covering Docker concepts.
Same remark for building Docker under Docker.
I do actually appreciate the DevStack and Docker building instructions but they should be in appendices, in my view.

OK, despite this curious organization of the books chapters the book is still an excellent read and goes into great detail about Docker basics.  Sometimes it feels like a reference manual specifying some docker CLI options without explanation of the terms used.  I've been using Docker for over a year now, nevertheless I did learn a few very useful tidbits from this section (in particularly the difference between load/save, import/export and also between ADD/COPY in Dockerfiles).

Often the "reference" section details a command-line option as 'exists since version 1.2" or similar.
It would be a good idea to state what is the baseline Docker version which is being documented as Docker is moving fast, Docker 1.5 should be out this week.

The book is very complete in the examples provided.  Whilst I've not tested all the examples they seem to make sense and exercise very many of Dockers capabilities of the docker client, the daemon, the REST api, interacting with Docker hub or a private registry, building images, linking containers and so on.

Chapter 3 covers setting resource limits (CPU share, memory usage, disk usage), mounting and sharing volumes,  choosing storage drivers, network configuration, linking containers directly or via the Ambassador pattern.

Chapter 4 covers automation via the REST api, and how to use service discovery with etcd to maintain connectivity across a group of containers dynamically (the Ambassador pattern does this but statically links containers causing the system to malfunction if containers die)

Chapter 4 covers orchestration also, mentioning the newly (December 2014) announced Docker Machine, Docker Swarm and Docker Compose but this was a bit light due to the recent nature of these beta products from Docker.

Chapter 4 terminates discussing security and best practices.
Looks like Chapter 4 could have been split into several chapters, by the way.

Chapter 5 is called "Friends of Docker" and covers Puppet and Chef configuration management tools and the use of apt-cacher.   Dokku is then used as an example (it's a mini-PaaS Heroku clone).

Then there is discussion of high-availability and scaling and examples of using CoreOS/etcd (for service discovery) and Fleet.

Overall this is an excellent book with lots of very useful well written content and is possibly the best Docker book out there right now.  I recommend it.