What makes a good MOOC

What's a MOOC?
Well if you've been living under a rock for the last year you may not have heard of Massively Open Online Courses, a new form of online education which is rapidly developing.

I took my first MOOC last September, when I was attracted to Coursera's Functional Programming in Scala course given by the languages' inventor Martin Odersky.  Since then I've taken another 10 or so on Coursera's platform.  I've also been drawn to other platforms, EdX, Class2Go (now merging with EdX), 10gen.com (EdX based), and Coursesites.

The quality of the courses and the hosting organizations varies but in general these courses have excellent content, supporting websites and evaluations.  My head is spinning from all that I've learnt these last 9 months in the domains of Languages (Scala, Python, Ruby, Racket, SMLNJ), Data (SQL, MongoDB, XML/XSLT/DTD/JSON), Design Patterns, CUDA, Arguing (yes, Think Again!), Web Apps (Ruby On Rails, TDD/BDD), Networking (SDN).

For the most part these are University level courses delivered by University professeurs via their chosen delivery platform.

It's interesting to compare my experiences to see what makes a good or bad MOOC.  I have to say that I haven't come across a bad MOOC but there are certainly some aspects which could be improved in some courses, and then there's my personal preferences ...

Creating a MOOC is a difficult task - I imagine - unsure if the course you're about to create will invite 1 or 10,000 students, no pre-selection of who will attend and so there's a need to satisfy the least and most experienced of students.  It's also a change of delivery mechanism for most lecturers from face to face delivery to digital delivery of videos and a minimum of written materials in the form of web pages and PDFs of lecture slides, course notes as well as a course Wiki, and course discussion forum.
It is of course necessary to evaluate the students and possibly award certificates.

Different methods of evaluation are used including

• multiple choice quizzes - sometimes with the possibility of several, even up to 100 retries
• peer evaluation - some aspects cannot be automated and so peer evaluation is used
• homeworks with machine validation - requiring automated marking

Creation of a course requires choosing one or more evaluation methods and possibly implementation of the machine based validation.  It's necessary that the MOOC platform is capable of handling a huge number of students concurrently, especially in the last hours before an assignment is due.  If calculations need to be performed on the MOOC platform, or delegated to some other cloud based platform it is necessary that the platform can handle the load peaks.  This can be quite demanding for some courses with a heavy work load such as CUDA parallel processing.

Students, even non-paying students, can be very unforgiving of platform problems, bad course materials or lack of communication from lecturers.  That said the forum discussions are generally very constructive - people are there to learn after all.

So what makes a good MOOC?

In my opinion ...

Site usability/navigation

Well, I've certainly appreciated the usability of the Coursera, Class2Go, EdX sites.  There are often many elements to a courses web site and so it's important not to get lost searching for information.

It can be very useful to have an overview of course progress / marks as well as approaching deadlines.

What I haven't seen yet is an amalgamation of course progress / deadlines on the same site across multiple courses - that would help me a lot !  I lose a lot of time passing between courses, between sites checking up on my deadlines.  Whilst I don't expect an amalgamation across platforms it would be nice if each platform provided an overall view across courses.

It's also nice to have one page which summarizes the download links for supporting materials ...

Supporting materials
MOOCs are generally centred around a series of lectures delivered as online videos.  These are generally downloadable by some means.   Generally a set of corresponding slides is provided in pdf format, this can be very effective to allow to quickly revise a whole course rather than sitting through N weeks of videos.  Some courses provide a set of reading notes as a set of PDFs which can be very useful.

I like to download the videos so that I can watch them when not online, when travelling.
What would really help is the means to automate downloading ... again a real time waster.
This can be facilitated by

• Ability to download all lecture PDFs with one click (for the week, for the whole course)
• Use of BitTorrent for downloading videos.  This would save a lot of people a lot of time and lighten up the load on networks and servers.   I suspect a lot of people just don't get it with BitTorrent, unable to dissociate the protocol with the idea of piracy which is something else.

Coursera's Programming Language course, given by Don Grossman, provided Videos (and downloadable subtitles), Lecture slides and separate notes (~ 30 pages each week).  This was a fantastic amount of material which greatly helped in what for me was a tough course.

Although the coverage of Class2Gos' Introduction to Database course was excellent and the overall assessment was excellent I was disappointed by the lack of information in PDF format and the lack of feedback on assignments.  It detracted from the pedagogy of the course, maybe some elitism from the course provider?  This is a personal opinion though as many people were very appreciative of this course.  I must say that the online quizzes and homeworks were extremely well done and this great helped learning.

Well structured assignments
Assignments should of course only cover the lecture material, or the lecture material should clearly indicate if other sources of information are required reading.  The purpose and instructions of the assignment must be clear.  Assignments should use the principles evoqued in class, stretching the student a little but not too much beyond what has been covered.

A bad MOOC leaves the student working for hours on an ill defined problem before submitting the result, sometimes just a number, to be marked instantly.
This can be a frustrating experience as the student doesn't know if he is 1% or 99%  complete, and when submitting a response will get a 0% or 100% score, possibly with no indication of what was expected.

A good MOOC tends to guide the student through the assignment with the means to measure progress.
Some courses have a very pedagogic approach preferring to validate at several steps and providing feedback on what might be wrong.

There might be a trade off between facilitating learning - thus providing the answers or lots of feedback - and a more elitist approach.

Deadlines and availability
Scheduling of assignment deadlines and availability of  weekly course material is also an issue.
Some courses allow self study, a nice to have option.  Udacity, which I haven't mentioned so far have courses which start at a particular date but which can be taken during the course dates or on a self-study basis.  Personally I work better with the pressure of deadlines though it's nice to have access to course materials ahead of time, or ahead of the course being scheduled - it's the case with Class2Go for example, or just availability of pre-course reading as is the case for Coursera's SDN course.

Some courses stick to a strict 1 week at a time policy with no overlap which can provide little leeway for assignments - assignments must be in by Monday, following weeks course materials and assignment details available on Tuesday.  No doubt easier to manage but tough as a student with a job, a family and weekend activities.  The ideal is to receive the course materials and assignments on a Friday for example with a submission date 10 days later, giving the flexibility of 2 weekends.

Staff input to Forum discussions
When problems occur the worst that can happen is lack of transparency from the course lecturers.  Students search for help in the discussion forums and may be helped by their peers.  But some problems can only be solved by the course providers and it's important that lecturers follow and contribute to the discussion.  If there are teething troubles with the initial offering of a course, problems getting the automated evaluation working correctly, under load for example then this should be recognized.  Lack of transparency, or flexibility will not be appreciated by the student unable to submit his assignment due to platform problems or inadequately described homeworks.

Choosing a MOOC
There are a number of useful sites allowing to find a MOOC or provide feedback on MOOCs.
I recently discovered MOOC-List which allows to search for available courses (in the Site Search box top right) across a large list of providers (bottom right).

Coursetalk provides student ratings of courses, your chance to give feedback and help others choose highly rated MOOCs.

It's helpful that the course provider provide clear information on course pre-requisites and the estimated workload.  But that workload is very subjective, depending upon experience, abilities and dedication to a course.

What's next?
Really a different subject and much more personal I pulled this out into a separate blog entry here