A real-time chronicle of a seasoned professor who just completed giving his second massively open online course.
The second running of my MOOC (massive open online course) Introduction to Mathematical Thinking ended recently. The basic stats were:
Total enrollment: 27,930
Number still active during final week of lectures: ca 4,000
Total submitting exam: 870
Number of students receiving a Statement of Accomplishment: 1,950
Number of students awarded a SoA with Distinction: 390
From my perspective, it went better than the first time, but this remains very much a research project, and will do for many more iterations. It is a research project with at least as many “Can we?” questions as “How do we?”
From the start, I took the viewpoint that, given the novelty of the MOOC platform, we need to examine the purpose, structure, and use of all the familiar educational elements: “lecture,” “quiz,” “assignment,” “discussion,” “grading,” “evaluation,” etc. All bets are off. Some changes to the way we use these elements might be minor, but on the other hand, some could be significant.
For instance, my course is not offered for any form of college credit. The goal is purely learning. This could be learning solely for its own sake, and many of my students approached it as such. On the other hand, as a course is basic analytic thinking and problem solving, with an emphasis on mathematical thinking in the second half of the course, it can clearly prepare a student to take (and hopefully do better in) future mathematics or STEM courses that do earn credit – and I have had students taking it with that goal in mind.
Separating learning from evaluation of what has been learned is enormously freeing, both to the instructor and to the student. In particular, evaluation of student work and the awarding of grades can be devoted purely to providing students with a useful (formative) indication of their progress, not a (summative) measure of their performance or ability.
To be sure, many of my students, conditioned by years of high stakes testing, have a hard time adjusting to the fact that a grade of 30% on a piece of work can be very respectable, indeed worth an A in many cases.
My typical response to students who lament their “low” grade is to say that their goal should be that a problem for which they struggle to get 30% in week 2 should be solvable for 80% or more by week 5 (say). And for problems they struggle with in week 8 (the final week of curriculum in my course), they should be able to do them more successfully if they take the course again the next time it is offered – something else that is possible in the brave new world of MOOCs. (Many of the students in my second offering of the course had attempted the first one a few months earlier.)
Incidentally, I think I have to make a comment regarding my statement above that the MOOC platform is novel. A number of commentators have observed that “online education is not new,” and they are right. But they miss the point that even this first generation of MOOC platforms represents a significant phase shift, not only in terms of the aggregate functionality but also the social and cultural context in which today’s MOOCs are being offered.
Regarding the context, not only have many of us grown accustomed to much of our interpersonal interaction being mediated by the internet, the vast majority of people under twenty now interact far more using social media than in person.
We could, of course, spend (I would say “waste”) our time debating whether or not this transition from physical space to cyberspace is a good thing. Personally, however, I think it is more productive to take steps to make sure it is – or at least ends up – a good thing. That means we need to take good education online, and we need to do so for the same reason that it’s important to embed good learning into video games.
The fact is, we have created for the new and future generations a world in which social media and video games are prevalent and attractive – just as earlier generations created worlds of books and magazines, and later mass broadcast media (radio, films, television) which were equally as widespread and attractive in their times. The media of any age are the ones through which we must pass on our culture and our cumulative learning. (See my other blog profkeithdevlin.org for my argument regarding learning in video games.)
Incidentally, I see the points I am making here (and will be making in future posts) as very much in alignment with, and definitely guided by, the views Sir Ken Robinson has expressed in a series of provocative lectures, 1, 2, 3.
Sir Ken’s thoughts influenced me a lot in my thinking about MOOCs. To be sure, there is much in the current version of my MOOC that looks very familiar. That is partly because of my academic’s professional caution, which tells me to proceed in small steps, starting from what I myself am familiar with; but in part also because the more significant changes I am presently introducing are the novel uses I am making (or trying to make) of familiar educational elements.
The design of my course was also heavily influenced by the expectation (more accurately a recognition, given how fast MOOCs are developing) that no single MOOC should see itself as the primary educational resource for a particular learning topic. Rather, those of us currently engaged in developing and offering MOOCs are, surely, creating resources that will be part of a vast smorgasbord from which people will pick and choose what they want or need at any particular time.
Given the way names get assigned and used, we may find we are stuck with the name MOOC (massive open online course), but a better term would be MOOR, for massive open online resource.
For basic, instructional learning, which makes up the bulk of K-12 mathematics teaching (wrongly in my view, but the US will only recognize that when virtually none of our home educated students are able to land the best jobs, which is about a generation away), that transition from course to resource has already taken place. YouTube is littered with short, instructional videos that teach people how to carry out certain procedures.
[By the way, I used the term “mathematical thinking” to describe my course, to distinguish it from the far more prevalent instructional math course that focuses on procedures. Students who did not recognize the distinction in the first three weeks, and approached the material accordingly, dropped out in droves in week four when they suddenly found themselves totally lost.]
By professional standards, many of the instructional video resources you can find on the Web (not just in mathematics but other subjects as well) are not very good, but that does not prevent them being very effective. As a professional mathematician and mathematics educator, I cringe when I watch a Khan Academy video, but millions find them of personal value. Analogously, in a domain where I am not an expert, bicycle mechanics, I watch Web videos to learn how to repair or tune my (high end) bicycles, and to assemble and disassemble my travel bike (a fairly complex process that literally has potential life and death consequences for me), and they serve my need, though I suspect a good bike mechanic would find much to critique in them. In both cases, mathematics and bicycle mechanics, some sites will iterate and improve, and in time they will dominate.
That last point, by the way, is another where many commentators miss the point. Something else that digital technologies and the Web make possible is rapid iteration guided by huge amounts of user feedback data – data obtained with great ease in almost real time.
In the days when products took a long time, and often considerable money, to plan and create, careful planning was essential. Today, we can proceed by a cycle of rapid prototypes. To be sure, it would be (in my view) unwise and unethical to proceed that way if a MOOC were being offered for payment or for some form of college credit, but for a cost-free, non-credit MOOC, learning on a platform that is itself under development, where the course designer is learning how to do it, can be in many ways a better learning experience than taking a polished product that has stood the test of time.
You don’t believe me? Consider this. Textbooks have been in regular use for over two thousand years, and millions of dollars have been poured into their development and production. Yet, take a look at practically any college textbook and ask yourself is you could, or would like to, learn from that source. In a system where the base level is the current college textbook and the bog-standard course built on it, the bar you have to reach with a MOOC to call it an improvement on the status quo is low indeed.
Again, Khan Academy provides the most dramatic illustration. Compared with what you will find in a good math classroom with a well trained teacher, it’s not good. But it’s a lot better than what is available to millions of students. More to the point, I know for a fact that Sal Khan is working on iterating from the starting point that caught Bill Gates’ attention, and has been for some time. Will he succeed? It hardly matters. (Well, I guess it does to Sal and his employees!) Someone will. (At least for a while, until someone else comes along and innovates a crucial step further.)
This, as I see it, is what, in general terms, is going on with MOOCs right now. We are experimenting. Needless to say – at least, it should be needless but there are worrying developments to the contrary – it would be unwise for any individual, any educational institution, or any educational district to make MOOCs (as courses) an important component of university education at this very early stage in their development. (And foolish to the point of criminality to take them into the K-12 system, but that’s a whole separate can of worms.)
Experimentation and rapid prototyping are fine in their place, but only when we all have more experience with them and have hard evidence of their efficacy (assuming they have such), should we start to think about giving them any critical significance in an educational system which (when executed properly) has served humankind well for several hundred years. Anyone who claims otherwise is probably trying to sell you something.
A final remark. I’m not saying that massive open online courses will go away. Indeed, I plan to continue offering mine – as a course – and I expect and hope many students will continue to take it as a complete course. I also expect that higher education institutions will increasingly incorporate MOOCs into their overall offerings, possibly for credit. (Stanford Online High School already offers a for-certificate course built around my MOOC.) So my use of the word “die” in the title involved a bit of poetic license.
But I believe my title is correct in its overall message. We already know from the research we’ve done at Stanford that only a minority of people enroll for a MOOC with the intention of taking it through to completion. (Though that “minority” can comprise several thousand students!) Most MOOC students already approach it as a resource, not a course! With an open online educational entity, it is the entire community of users that ultimately determines what it primarily is and how it fits in the overall educational landscape. According to the evidence, they already have, thereby giving us a new (and more accurate) MOOC mantra: resources, not courses. (Even when they are courses and when some people take them as such.)
In the coming posts to this blog, I’ll report on the changes I made in the second version of my MOOC, reflect on how things turned out, and speculate about the changes I am thinking of making in version 3, which is scheduled to start in September. First topic up will be peer evaluation – something that I regard as key to the success of a MOOC on mathematical thinking.
Those of us in education are fortunate to be living in a time where there is so much potential for change. The last time anything happened on this scale in the world of education was the invention of the printing press in the Fifteenth Century. As you can probably tell, I am having a blast.
To be continued …
MOOCtalk 
Thank You for trying to bring change to education via the power of the electronics/self learning via utube age. Perhaps a U of Tube degree will be the offspring of this endeavor.
The MOOC/MOOR distinction strikes me as extremely important, because I believe there is a strong case to be made that MOOCs are not promising as a delivery method for online learning.
In particular, it strikes me that:
(1) On the one hand, since little faculty-student interaction occurs in MOOCs, they will never be a viable “complete” form of online learning delivery.
(2) On the other hand, the “star-power” lectures and other learning resources they offer may certainly be useful in both online and offline classes as online resources a la the flipped classroom.
In my above comment, I have in mind the question of whether MOOCs are appropriate as a sole method for delivering credit-bearing higher education. I think the lack of intensive professor-student interaction is not a fatal flaw for those cases where MOOCs are used to promote lifelong learning outside a credit-bearing degree program.
Within degree programs, the MOOR model strikes me as promising even if I am right that, ultimately, the MOOC model is inadequate.
I thought MOOR already stood for Massive Open Online Research (the crowdsourced, citizen-type science). Liked your blog, though.
MOOCS are for type “A” students.
“By the way, I used the term “mathematical thinking” to describe my course, to distinguish it from the far more prevalent instructional math course that focuses on procedures. Students who did not recognize the distinction in the first three weeks, and approached the material accordingly, dropped out in droves in week four when they suddenly found themselves totally lost.”
As someone who completed the course and organised tutorials in London for others involved, I can say that the reason people dropped out between weeks 3 and 4 is to do with the topic of ‘implication’, not for the reason you cite. Students were given no reason to prefer the mathematical definition to their own everyday use and intuitions and without that couldn’t see why they should submit to the discipline/authority of the course. Those of us with a bit more experience were left with a dilemma: did we become volunteer tutors introducing material from outside the course to address that gap? or decide that we didn’t really have the time or inclination to act in that way?
Otherwise I would agree with the argument about seeing this course as a very useful resource (by far the best mooc I have tried so far). Peer-grading the examination left me wanting a flag for possible plagiarism in 2 out of 3 cases – another point which undermines attempts to push coursera courses as credit-bearing alternatives. I see them as fitting into the continuing education short course provision.
Andrew, Thanks for the feedback. I think what you say about implication actually supports my point. The principal focus in the first three weeks was how the ambiguity of everyday language (which is a plus in everyday communication about the world, as it enables us to use a compact language to discuss a great range of different issues) becomes problematic in mathematics, where language actually creates the world being discussed. A student who had really grasped that crucial point — and there was really no other point in the entire first three weeks — should have been ready to appreciate the need for making a specific choice regarding the meaning of the word “implies” that is non-ambiguous and everywhere defined. The (standard) rationale I gave for the definition adopted will seem arbitrary only if not approached in the context of a problem and a need. What a “mathematical thinker” should do in those cases is try alternatives and see why they ultimately prove to be not fruitful. Among the many students who did not drop out, many subsequently went on the forums to help others, and later in the course the ones who made the transition to “mathematical thinker” reported on their appreciation for making the transition. Students who — ignoring my repeatedly stressing the purpose of the course — nevertheless looked for rules to follow and procedures to apply, and did not view it as an opportunity to collectively reflect on matters and develop a new way of thinking, suddenly found themselves lost. The “definition” of implication I gave is a fatal problem only if viewed as an “arbitrary rule from on high,” rather than a road sign pointing to the optimal solution that would guide them in their own thinking.
In a course trying to develop a new way of thinking, there has to come a point when the students first try to engage that kind of thinking. The problems associated with implication provides one convenient way to force that initial engagement — one that can be arrived at in a couple of weeks — but I could have chosen another. I think it is an inescapable fact of human psychology that it has to be forced, and as a consequence, many will simply drop out. After all, I spent three weeks telling folks what was about to happen, but when it did, many were not prepared.
What I would suggest for a course where there is no ‘entry and exit’ cost, is that you show what can be done with the new way of thinking at an earlier stage (perhaps more detail on some of the examples of real world application you cite?). Even if it is supplementary material outlining the power of that understanding of implication. Everyone in our London group was a graduate and were there for the ‘thinking’, not procedures.
The members who dropped out of that group could ‘do it’, (after a fair bit of real life tutorial support), but they just found it artificial and didn’t really have a frame in which to assess its potential power at that stage. Perhaps I should have dug out some stuff from Kahnemann …
I’m not saying it’s a ‘fatal problem’, but I think many would stick if they could get better insights into the benefits at that stage. Reiterating the ‘purpose’ may not be sufficient.
Often the pedagogical problem is engaging the students who don’t speak up in class and there appears to be a silent group who dislike and don’t use the online forums, which means your data is subject to bias.
With reference to your point that it has to be ‘forced’, I’d point back to the absence of ‘entry/exit’ costs and recall Aesop’s fable about the north wind and the sun.
Anyway, this is intended as constructive: I found the course extremely useful (I’d be interested in one on set theory …). I probably should have underscored that before.
That’s very helpful feedback. Thanks!
Things are changing too fast. Now 10 state universities bought online courses from Coursera at $36 per course. That is what I call a revolution. Up to now MOOCs were just an experiment. Now it is real life. How many students will take MOOCs at a small fee for credits and degrees?
One thing I am sure, SUNY will be very rich. Globally a potential 200,000,000 HE students will flow to SUNY to get a degree at $1,440 without leaving their country. SUNY will be very rich.
1.25 million students of 10 state universities will be very happy too to pay less tuitions now.
I still say good MOOCs from non profits will stay alive. Rest will die.
Duncan should look up these 10 state universities very carefully. If I were him I would cut loans and grants to every state university if they do not adopt at least 10 courses from edX and Coursera.
dear Keith.. if/when you have time.. would love a smidge of an expansion on this sentence:
I think it is an inescapable fact of human psychology that it has to be forced, and as a consequence, many will simply drop out.
grazie.
Monika, Good question. My experience in teaching, and that of colleagues I have talked to, is that explaining in advance what the difficulties are and how to avoid them has little effect. Only after a student has gone wrong and is faced with having to figure out what went wrong do they adapt and adopt an approach that works. We rarely learn from what we are told; but we almost always learn from our mistakes.
Student who are discouraged by making mistakes will likely drop out. Those of us who realize that making mistakes is how we learn are likely to stick with it.
Dear Keith,
I have recently enjoyed and completed your MOOC, I found it to be a mind-expanding experience leading to tangible benefits in terms of my thinking and positive knock-ons in many areas of life.
I am also educating my children at home, I hope that they will acquire effective thinking and mathematical skills as they mature in this environment.
In view of this, I am wondering whether you could clarify your thoughts on the following statement from your post:
“but the US will only recognize that when virtually none of our home educated students are able to land the best jobs, which is about a generation away”
Thank you.
Ah, an unfortunate choice of words. By “home educated” I meant “US educated”, and even more precisely, “educated by the US public education system”. I was certainly not thinking about home schooling.
Thank you for clarifying this, and for energetically furthering debate and progress in education and learning.
I look forward to watching how MOOC(R)s evolve over time.
Personally I gained a lot from taking your MOOC on as a course,rather than as simply a resource, given that the weekly deadlines ensured consistent time allocation to the coursework.
All the best with future editions of MathThink.
I found the first edition of the course just as it was wrapping up, and snuck in to peek around. The second running of the course completed not long ago got smacked down by my real life commitments, but I liked what I saw.
In terms of the interface, I found a lot of advantages generated by the student body (a very lucrative example being the script that pulled all the lecture videos from YouTube into files you could use anywhere, anytime, without needing an Internet connection).
I think one key miss I found was the transition from discussion & introduction into mathematical formulas in what seemed the blink of an eye – guess it’s been too long since I did maths to make the jump. A big focus of the MOOC planning and execution seemed to be the platform – and rightly so! It can certainly make or break the success of just getting to the information, let alone comprehending it and understanding the concepts.
It was Coursera interface (and similarly, the MITx one) that I was reminded of when I recently tried the following site [apologies if links are not preferred] – I was initially thrown by their forum/Q&A panels, until I realised they were *question-centric* …which is exactly what happens in class. You don’t want to trawl through questions of the whole course (unless there are general questions about course intent) — rather, I found that if I had a question with the current panel, then it was extremely useful to read and contribute to discussion about it right then and there.
http://www.codecademy.com/courses/python-beginner-c7VZg/0/2?curriculum_id=4f89dab3d788890003000096
The pyramid structure of a single html panel -> many panels -> sections -> chapters -> courses also seems to be more student-focused and support the reality of leave anytime / return to exactly where you were that online learning has to react to.
Anyway, just thought I’d share some thoughts. Will continue to keep an eye on Coursera in the future.